2024-04-25T20:49:03.645 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T20:49:03.646 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --verbose --name leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657 -- /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/orig.config.yaml 2024-04-25T20:49:03.706 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T20:49:03.797 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7673657' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2919 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNJcB+LGTquhUgfcYScSYctMkQflWtv9OmmG955gaXFaCeV/WFZvRVYHzah4XuSi3HEVTb/mTjR4JWpoTrEqw50= smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0pk+bi6PCPN60VVLMf5Y+r+81DV46nwg5kGVhebEI/83XGHKBeRzl9BR4EdLAQX2E0OsA3K9GrYmeeV6LUCDw= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGeDXI6Ao+Q1yWAxF35Rn+d4hj9ar4ZPtxTIdaigHeQZOQSZ3fGJ3h78Ixa4gA6xNV9ND+92wRnTsklPBtMF8So= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_19:59:35 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T20:49:03.798 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa; will attempt to use it 2024-04-25T20:49:03.798 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks 2024-04-25T20:49:03.799 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T20:49:03.800 INFO:teuthology.task.internal:Checking packages... 2024-04-25T20:49:03.823 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T20:49:03.824 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T20:49:03.824 INFO:teuthology.packaging:ref: None 2024-04-25T20:49:03.824 INFO:teuthology.packaging:tag: None 2024-04-25T20:49:03.824 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T20:49:03.824 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:49:03.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T20:49:03.981 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T20:49:03.982 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T20:49:03.989 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T20:49:03.989 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T20:49:03.999 INFO:teuthology.task.internal:Saving configuration 2024-04-25T20:49:04.019 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T20:49:04.026 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T20:49:04.051 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:43:55.220383', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNJcB+LGTquhUgfcYScSYctMkQflWtv9OmmG955gaXFaCeV/WFZvRVYHzah4XuSi3HEVTb/mTjR4JWpoTrEqw50='} 2024-04-25T20:49:04.076 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:43:55.218546', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0pk+bi6PCPN60VVLMf5Y+r+81DV46nwg5kGVhebEI/83XGHKBeRzl9BR4EdLAQX2E0OsA3K9GrYmeeV6LUCDw='} 2024-04-25T20:49:04.100 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:43:55.221982', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGeDXI6Ao+Q1yWAxF35Rn+d4hj9ar4ZPtxTIdaigHeQZOQSZ3fGJ3h78Ixa4gA6xNV9ND+92wRnTsklPBtMF8So='} 2024-04-25T20:49:04.100 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T20:49:04.107 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T20:49:04.107 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T20:49:04.107 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T20:49:04.107 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T20:49:04.215 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa844daa0d0>, signals=[15]) 2024-04-25T20:49:04.216 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T20:49:04.222 INFO:teuthology.task.internal:Opening connections... 2024-04-25T20:49:04.222 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T20:49:04.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:49:04.302 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-04-25T20:49:04.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:49:04.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2024-04-25T20:49:04.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:49:04.457 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T20:49:04.464 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-25T20:49:04.483 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-25T20:49:04.483 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-25T20:49:04.541 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-25T20:49:04.541 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-25T20:49:04.541 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:49:04.542 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:49:04.543 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-25T20:49:04.565 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-04-25T20:49:04.583 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-04-25T20:49:04.583 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-04-25T20:49:04.639 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="CentOS Stream" 2024-04-25T20:49:04.639 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="9" 2024-04-25T20:49:04.639 INFO:teuthology.orchestra.run.smithi098.stdout:ID="centos" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="9" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:ANSI_COLOR="0;31" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:49:04.640 INFO:teuthology.orchestra.run.smithi098.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:49:04.640 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-04-25T20:49:04.665 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2024-04-25T20:49:04.684 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2024-04-25T20:49:04.684 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2024-04-25T20:49:04.742 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="9" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="9" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2024-04-25T20:49:04.743 INFO:teuthology.orchestra.run.smithi177.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:49:04.744 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:49:04.744 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:49:04.744 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:49:04.744 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:49:04.744 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:49:04.744 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2024-04-25T20:49:04.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T20:49:04.776 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T20:49:04.782 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T20:49:04.782 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:49:04.784 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:49:04.786 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:49:04.802 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T20:49:04.811 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T20:49:04.811 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:49:04.842 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:49:04.844 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:49:04.947 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T20:49:05.019 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T20:49:05.033 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T20:49:05.034 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T20:49:05.034 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T20:49:05.034 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:49:05.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:05.052 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-25T20:49:05.110 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-25T20:49:05.111 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-437.el9.x86_64 2024-04-25T20:49:05.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-04-25T20:49:07.630 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 8.8 MB/s | 8.0 MB 00:00 2024-04-25T20:49:09.423 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-04-25T20:49:14.074 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-25T20:49:15.764 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T20:49:17.116 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T20:49:22.399 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-25T20:49:24.372 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T20:49:24.372 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T20:49:24.461 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T20:49:24.462 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-25T20:49:24.462 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T20:49:24.560 DEBUG:teuthology.orchestra.run.smithi037:> echo no | sudo yum reinstall kernel || true 2024-04-25T20:49:25.081 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:49:22 PM UTC. 2024-04-25T20:49:25.209 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T20:49:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T20:49:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:Operation aborted. 2024-04-25T20:49:25.274 INFO:teuthology.orchestra.run.smithi037.stdout:Is this ok [y/N]: 2024-04-25T20:49:25.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum reinstall -y kernel || true 2024-04-25T20:49:25.791 INFO:teuthology.orchestra.run.smithi037.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:49:22 PM UTC. 2024-04-25T20:49:25.912 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalling: 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:49:25.913 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 255 k 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 0 2024-04-25T20:49:25.914 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T20:49:26.309 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-25T20:49:26.310 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:49:26.310 INFO:teuthology.orchestra.run.smithi037.stdout:Total 643 kB/s | 255 kB 00:00 2024-04-25T20:49:26.330 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T20:49:26.534 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T20:49:26.534 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T20:49:26.539 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T20:49:26.539 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T20:49:26.676 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T20:49:26.735 INFO:teuthology.orchestra.run.smithi037.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:49:26.735 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:49:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:49:27.368 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout:Reinstalled: 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:49:28.056 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T20:49:28.214 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T20:49:28.261 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T20:49:28.261 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T20:49:28.261 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T20:49:28.262 DEBUG:teuthology.parallel:result is None 2024-04-25T20:49:28.262 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T20:49:28.269 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T20:49:28.270 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:49:28.272 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:49:28.274 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:49:28.291 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T20:49:28.298 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T20:49:28.308 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T20:49:28.308 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:49:28.326 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:49:28.331 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:49:28.433 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T20:49:28.440 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T20:49:28.440 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:49:28.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:28.458 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:49:28.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:28.474 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:49:28.476 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:49:28.500 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:49:28.521 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.541 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.543 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.563 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.566 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.579 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:49:28.581 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T20:49:28.588 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T20:49:28.589 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:49:28.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:49:28.593 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:49:28.619 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T20:49:28.627 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T20:49:28.627 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:49:28.655 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:49:28.660 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:49:28.676 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:28.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:28.800 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:28.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:28.921 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:49:28.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:49:28.985 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:29.022 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:29.082 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:29.163 DEBUG:teuthology.orchestra.run.smithi098:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:29.208 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:49:29.208 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:49:29.274 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:29.314 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:49:29.371 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:29.448 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:49:29.498 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:49:29.498 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:49:29.564 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-25T20:49:29.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-04-25T20:49:29.569 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2024-04-25T20:49:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:49:29.612 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:49:29.643 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:49:30.129 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T20:49:30.137 INFO:teuthology.task.internal:Starting timer... 2024-04-25T20:49:30.137 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T20:49:30.156 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T20:49:30.164 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T20:49:30.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-25T20:49:30.246 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-25T20:49:30.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo service auditd rotate 2024-04-25T20:49:30.308 INFO:teuthology.orchestra.run.smithi098.stdout:Rotating logs: 2024-04-25T20:49:30.309 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2024-04-25T20:49:30.365 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2024-04-25T20:49:30.367 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T20:49:30.367 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-25T20:49:30.392 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-25T20:49:30.393 DEBUG:teuthology.orchestra.run.smithi098:> /usr/sbin/getenforce 2024-04-25T20:49:30.419 INFO:teuthology.orchestra.run.smithi098.stdout:Permissive 2024-04-25T20:49:30.420 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2024-04-25T20:49:30.463 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2024-04-25T20:49:30.464 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi098.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2024-04-25T20:49:30.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:49:30.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:30.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:49:30.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:30.520 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:49:30.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:49:30.549 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T20:49:30.549 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:49:30.575 DEBUG:teuthology.orchestra.run.smithi098:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:49:30.602 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:49:30.633 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T20:49:30.649 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T20:49:30.656 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T20:49:30.793 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T20:49:30.810 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T20:49:30.811 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi098.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T20:53:18.351 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2024-04-25T20:53:18.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T20:53:18.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:53:18.430 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-25T20:53:18.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T20:53:18.514 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-25T20:53:18.515 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:53:18.593 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-04-25T20:53:18.631 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-04-25T20:53:18.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-25T20:53:18.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:53:18.709 DEBUG:teuthology.orchestra.run.smithi177:> true 2024-04-25T20:53:18.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2024-04-25T20:53:18.786 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T20:53:18.798 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T20:53:18.799 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:53:18.799 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:53:18.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:53:18.801 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:53:18.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:53:18.804 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:53:18.829 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:53:18.830 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:53:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:53:18.844 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:53:18.855 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:53:18.867 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: ntpd: command not found 2024-04-25T20:53:18.868 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-25T20:53:18.869 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:53:18.876 INFO:teuthology.orchestra.run.smithi098.stdout:506 Cannot talk to daemon 2024-04-25T20:53:18.879 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-25T20:53:18.889 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:53:18.893 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2024-04-25T20:53:18.902 INFO:teuthology.orchestra.run.smithi098.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:53:18.903 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2024-04-25T20:53:18.906 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:53:18.917 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:53:18.930 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:53:18.961 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:53:18.963 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.965 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.968 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.990 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.994 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:53:18.995 INFO:teuthology.run_tasks:Running task install... 2024-04-25T20:53:19.003 DEBUG:teuthology.task.install:project ceph 2024-04-25T20:53:19.003 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T20:53:19.003 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T20:53:19.003 INFO:teuthology.task.install:Using flavor: default 2024-04-25T20:53:19.009 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T20:53:19.009 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T20:53:19.009 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:53:19.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:53:19.012 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:53:19.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:53:19.013 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:53:19.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:53:19.168 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:53:19.168 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:53:19.249 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:53:19.249 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:53:19.255 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:53:19.255 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:53:19.340 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:53:19.340 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:53:19.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:53:19.358 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:53:19.358 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:53:19.358 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:53:19.368 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:53:19.368 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:53:19.369 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:53:19.370 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:53:19.370 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:53:19.381 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:53:19.382 DEBUG:teuthology.orchestra.run.smithi098:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:53:19.395 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:53:19.395 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:53:19.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:53:19.446 DEBUG:teuthology.orchestra.run.smithi098:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:53:19.460 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:53:19.510 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:53:19.519 DEBUG:teuthology.orchestra.run.smithi098:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:53:19.533 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:53:19.574 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-25T20:53:19.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-25T20:53:19.581 INFO:teuthology.orchestra.run.smithi098.stdout:check_obsoletes = 1 2024-04-25T20:53:19.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum clean all 2024-04-25T20:53:19.596 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2024-04-25T20:53:19.597 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2024-04-25T20:53:19.805 INFO:teuthology.orchestra.run.smithi098.stdout:48 files removed 2024-04-25T20:53:19.810 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-25T20:53:19.822 INFO:teuthology.orchestra.run.smithi177.stdout:48 files removed 2024-04-25T20:53:19.832 DEBUG:teuthology.orchestra.run.smithi098:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:53:19.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:53:19.848 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:53:20.519 INFO:teuthology.orchestra.run.smithi098.stdout:ceph packages for x86_64 195 kB/s | 83 kB 00:00 2024-04-25T20:53:20.607 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 167 kB/s | 83 kB 00:00 2024-04-25T20:53:20.625 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 163 kB/s | 83 kB 00:00 2024-04-25T20:53:20.920 INFO:teuthology.orchestra.run.smithi098.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T20:53:21.050 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-25T20:53:21.101 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-25T20:53:21.270 INFO:teuthology.orchestra.run.smithi098.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T20:53:21.467 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-25T20:53:21.528 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T20:53:22.421 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-25T20:53:22.545 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-04-25T20:53:22.614 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-04-25T20:53:24.404 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-25T20:53:24.589 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T20:53:24.949 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T20:53:28.596 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T20:53:28.781 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T20:53:29.150 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-25T20:53:30.094 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-25T20:53:30.296 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-25T20:53:30.748 INFO:teuthology.orchestra.run.smithi098.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T20:53:32.334 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T20:53:32.469 INFO:teuthology.orchestra.run.smithi098.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T20:53:32.882 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 21 MB 00:02 2024-04-25T20:53:37.473 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 51 kB/s | 1.7 kB 00:00 2024-04-25T20:53:37.583 INFO:teuthology.orchestra.run.smithi098.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T20:53:38.004 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T20:53:39.692 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.693 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:53:39.705 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.706 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:53:39.707 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:53:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:53:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:53:39.712 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:53:39.745 INFO:teuthology.orchestra.run.smithi098.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.746 INFO:teuthology.orchestra.run.smithi098.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.756 INFO:teuthology.orchestra.run.smithi098.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:53:39.757 INFO:teuthology.orchestra.run.smithi098.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:53:39.758 INFO:teuthology.orchestra.run.smithi098.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:53:39.759 INFO:teuthology.orchestra.run.smithi098.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:53:39.760 INFO:teuthology.orchestra.run.smithi098.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:53:39.763 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:53:39.764 INFO:teuthology.orchestra.run.smithi098.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:53:39.777 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-25T20:53:39.786 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T20:53:39.786 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:53:39.787 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:53:39.788 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:53:39.789 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:53:39.790 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:53:39.791 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:53:39.792 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:53:39.793 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:53:39.794 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:53:39.795 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:53:39.796 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:53:39.797 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:53:39.798 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-25T20:53:39.799 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-25T20:53:39.824 INFO:teuthology.orchestra.run.smithi098.stdout:Dependencies resolved. 2024-04-25T20:53:39.844 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-04-25T20:53:39.844 INFO:teuthology.orchestra.run.smithi098.stdout: Package Arch Version Repository Size 2024-04-25T20:53:39.844 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-04-25T20:53:39.844 INFO:teuthology.orchestra.run.smithi098.stdout:Installing: 2024-04-25T20:53:39.844 INFO:teuthology.orchestra.run.smithi098.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:53:39.845 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:53:39.846 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrading: 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:53:39.847 INFO:teuthology.orchestra.run.smithi098.stdout:Installing dependencies: 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:53:39.848 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:53:39.849 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:53:39.850 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:53:39.851 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:53:39.852 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:53:39.853 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:53:39.854 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:53:39.855 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout:Installing weak dependencies: 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:53:39.856 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction Summary 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout:======================================================================================= 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout:Install 148 Packages 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout:Upgrade 4 Packages 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout:Total download size: 222 M 2024-04-25T20:53:39.857 INFO:teuthology.orchestra.run.smithi098.stdout:Downloading Packages: 2024-04-25T20:53:40.197 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:40.198 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:53:40.209 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:53:40.210 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:53:40.210 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:53:40.211 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:53:40.213 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:53:40.215 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:53:40.216 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:53:40.276 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2024-04-25T20:53:40.305 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-04-25T20:53:40.305 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2024-04-25T20:53:40.305 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-04-25T20:53:40.305 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:53:40.306 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:53:40.307 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:53:40.308 INFO:teuthology.orchestra.run.smithi177.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:53:40.309 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:53:40.310 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:53:40.311 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:53:40.312 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:53:40.313 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:53:40.314 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:53:40.315 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout:Installing weak dependencies: 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout:======================================================================================= 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout:Install 148 Packages 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 4 Packages 2024-04-25T20:53:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:53:40.317 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 222 M 2024-04-25T20:53:40.317 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2024-04-25T20:53:40.751 INFO:teuthology.orchestra.run.smithi098.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-25T20:53:41.100 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-25T20:53:41.126 INFO:teuthology.orchestra.run.smithi098.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-25T20:53:41.218 INFO:teuthology.orchestra.run.smithi098.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.5 MB/s | 145 kB 00:00 2024-04-25T20:53:41.660 INFO:teuthology.orchestra.run.smithi098.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.8 MB/s | 5.4 MB 00:01 2024-04-25T20:53:41.708 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T20:53:41.755 INFO:teuthology.orchestra.run.smithi177.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T20:53:41.825 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.2 MB/s | 145 kB 00:00 2024-04-25T20:53:42.027 INFO:teuthology.orchestra.run.smithi098.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.9 MB/s | 2.4 MB 00:00 2024-04-25T20:53:42.069 INFO:teuthology.orchestra.run.smithi098.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-25T20:53:42.525 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 2.4 MB 00:00 2024-04-25T20:53:42.603 INFO:teuthology.orchestra.run.smithi098.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 10 MB/s | 21 MB 00:02 2024-04-25T20:53:42.900 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-25T20:53:43.842 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.0 MB/s | 4.7 MB 00:00 2024-04-25T20:53:43.905 INFO:teuthology.orchestra.run.smithi177.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 414 kB/s | 890 kB 00:02 2024-04-25T20:53:44.003 INFO:teuthology.orchestra.run.smithi098.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 4.7 MB 00:01 2024-04-25T20:53:44.128 INFO:teuthology.orchestra.run.smithi098.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 201 kB/s | 25 kB 00:00 2024-04-25T20:53:44.303 INFO:teuthology.orchestra.run.smithi098.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 6.2 MB/s | 11 MB 00:01 2024-04-25T20:53:44.339 INFO:teuthology.orchestra.run.smithi177.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 334 kB/s | 145 kB 00:00 2024-04-25T20:53:44.395 INFO:teuthology.orchestra.run.smithi098.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-25T20:53:44.470 INFO:teuthology.orchestra.run.smithi098.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-25T20:53:44.546 INFO:teuthology.orchestra.run.smithi098.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-25T20:53:44.614 INFO:teuthology.orchestra.run.smithi177.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-25T20:53:44.713 INFO:teuthology.orchestra.run.smithi098.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.4 MB/s | 747 kB 00:00 2024-04-25T20:53:44.805 INFO:teuthology.orchestra.run.smithi098.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T20:53:44.897 INFO:teuthology.orchestra.run.smithi098.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.3 MB/s | 126 kB 00:00 2024-04-25T20:53:45.064 INFO:teuthology.orchestra.run.smithi098.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.9 MB/s | 500 kB 00:00 2024-04-25T20:53:45.231 INFO:teuthology.orchestra.run.smithi098.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 5.3 MB/s | 17 MB 00:03 2024-04-25T20:53:45.321 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-25T20:53:45.323 INFO:teuthology.orchestra.run.smithi098.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 493 kB/s | 45 kB 00:00 2024-04-25T20:53:45.414 INFO:teuthology.orchestra.run.smithi177.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 1.5 MB 00:00 2024-04-25T20:53:45.415 INFO:teuthology.orchestra.run.smithi098.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T20:53:45.523 INFO:teuthology.orchestra.run.smithi098.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T20:53:45.640 INFO:teuthology.orchestra.run.smithi098.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 320 kB 00:00 2024-04-25T20:53:45.749 INFO:teuthology.orchestra.run.smithi098.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 298 kB 00:00 2024-04-25T20:53:45.849 INFO:teuthology.orchestra.run.smithi098.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 995 kB/s | 99 kB 00:00 2024-04-25T20:53:46.000 INFO:teuthology.orchestra.run.smithi098.stdout:(25/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.7 MB/s | 5.4 MB 00:00 2024-04-25T20:53:46.025 INFO:teuthology.orchestra.run.smithi098.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 486 kB/s | 85 kB 00:00 2024-04-25T20:53:46.116 INFO:teuthology.orchestra.run.smithi098.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-25T20:53:46.192 INFO:teuthology.orchestra.run.smithi098.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 457 kB/s | 34 kB 00:00 2024-04-25T20:53:46.201 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 7.1 MB/s | 17 MB 00:02 2024-04-25T20:53:46.267 INFO:teuthology.orchestra.run.smithi098.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-25T20:53:46.351 INFO:teuthology.orchestra.run.smithi098.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 3.1 MB 00:00 2024-04-25T20:53:46.409 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 21 MB 00:05 2024-04-25T20:53:46.434 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 108 kB/s | 25 kB 00:00 2024-04-25T20:53:46.535 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 225 kB 00:00 2024-04-25T20:53:46.635 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T20:53:46.735 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 317 kB/s | 32 kB 00:00 2024-04-25T20:53:46.810 INFO:teuthology.orchestra.run.smithi098.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 6.5 MB/s | 3.5 MB 00:00 2024-04-25T20:53:46.894 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 747 kB 00:00 2024-04-25T20:53:46.918 INFO:teuthology.orchestra.run.smithi098.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-25T20:53:47.011 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-25T20:53:47.222 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.3 MB/s | 126 kB 00:00 2024-04-25T20:53:47.222 INFO:teuthology.orchestra.run.smithi098.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 423 kB/s | 49 kB 00:00 2024-04-25T20:53:47.222 INFO:teuthology.orchestra.run.smithi098.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 141 kB/s | 16 kB 00:00 2024-04-25T20:53:47.236 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.7 MB/s | 500 kB 00:00 2024-04-25T20:53:47.286 INFO:teuthology.orchestra.run.smithi098.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.0 MB/s | 273 kB 00:00 2024-04-25T20:53:47.428 INFO:teuthology.orchestra.run.smithi098.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-25T20:53:47.478 INFO:teuthology.orchestra.run.smithi098.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 367 kB/s | 18 kB 00:00 2024-04-25T20:53:47.520 INFO:teuthology.orchestra.run.smithi098.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.2 MB/s | 762 kB 00:00 2024-04-25T20:53:47.556 INFO:teuthology.orchestra.run.smithi177.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 752 kB/s | 2.4 MB 00:03 2024-04-25T20:53:47.737 INFO:teuthology.orchestra.run.smithi098.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 311 kB/s | 80 kB 00:00 2024-04-25T20:53:47.764 INFO:teuthology.orchestra.run.smithi177.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-25T20:53:47.829 INFO:teuthology.orchestra.run.smithi098.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-25T20:53:47.906 INFO:teuthology.orchestra.run.smithi098.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-25T20:53:47.955 INFO:teuthology.orchestra.run.smithi098.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T20:53:47.978 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.2 MB/s | 5.4 MB 00:00 2024-04-25T20:53:48.039 INFO:teuthology.orchestra.run.smithi098.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-04-25T20:53:48.045 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 679 kB/s | 45 kB 00:00 2024-04-25T20:53:48.064 INFO:teuthology.orchestra.run.smithi098.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 986 kB/s | 106 kB 00:00 2024-04-25T20:53:48.089 INFO:teuthology.orchestra.run.smithi098.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T20:53:48.104 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.2 MB/s | 135 kB 00:00 2024-04-25T20:53:48.114 INFO:teuthology.orchestra.run.smithi098.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T20:53:48.140 INFO:teuthology.orchestra.run.smithi098.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T20:53:48.156 INFO:teuthology.orchestra.run.smithi177.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.1 MB/s | 21 MB 00:06 2024-04-25T20:53:48.165 INFO:teuthology.orchestra.run.smithi098.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T20:53:48.199 INFO:teuthology.orchestra.run.smithi098.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T20:53:48.204 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.6 MB/s | 161 kB 00:00 2024-04-25T20:53:48.224 INFO:teuthology.orchestra.run.smithi098.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T20:53:48.231 INFO:teuthology.orchestra.run.smithi177.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 336 kB/s | 25 kB 00:00 2024-04-25T20:53:48.241 INFO:teuthology.orchestra.run.smithi098.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-04-25T20:53:48.266 INFO:teuthology.orchestra.run.smithi098.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T20:53:48.291 INFO:teuthology.orchestra.run.smithi098.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-04-25T20:53:48.313 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 320 kB 00:00 2024-04-25T20:53:48.375 INFO:teuthology.orchestra.run.smithi098.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T20:53:48.388 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-25T20:53:48.400 INFO:teuthology.orchestra.run.smithi098.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 169 kB/s | 18 kB 00:00 2024-04-25T20:53:48.426 INFO:teuthology.orchestra.run.smithi098.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T20:53:48.451 INFO:teuthology.orchestra.run.smithi098.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T20:53:48.463 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T20:53:48.476 INFO:teuthology.orchestra.run.smithi098.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T20:53:48.510 INFO:teuthology.orchestra.run.smithi098.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T20:53:48.530 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 85 kB 00:00 2024-04-25T20:53:48.535 INFO:teuthology.orchestra.run.smithi098.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T20:53:48.569 INFO:teuthology.orchestra.run.smithi098.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 442 kB/s | 25 kB 00:00 2024-04-25T20:53:48.594 INFO:teuthology.orchestra.run.smithi098.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-25T20:53:48.620 INFO:teuthology.orchestra.run.smithi098.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T20:53:48.645 INFO:teuthology.orchestra.run.smithi098.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T20:53:48.670 INFO:teuthology.orchestra.run.smithi098.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 710 kB/s | 35 kB 00:00 2024-04-25T20:53:48.804 INFO:teuthology.orchestra.run.smithi098.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-04-25T20:53:48.913 INFO:teuthology.orchestra.run.smithi098.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T20:53:48.947 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.4 MB/s | 3.1 MB 00:00 2024-04-25T20:53:48.954 INFO:teuthology.orchestra.run.smithi098.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T20:53:48.996 INFO:teuthology.orchestra.run.smithi098.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T20:53:49.022 INFO:teuthology.orchestra.run.smithi098.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T20:53:49.023 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.2 MB/s | 170 kB 00:00 2024-04-25T20:53:49.115 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 374 kB/s | 34 kB 00:00 2024-04-25T20:53:49.190 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-25T20:53:49.197 INFO:teuthology.orchestra.run.smithi098.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T20:53:49.231 INFO:teuthology.orchestra.run.smithi098.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-25T20:53:49.256 INFO:teuthology.orchestra.run.smithi098.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T20:53:49.415 INFO:teuthology.orchestra.run.smithi098.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-04-25T20:53:49.440 INFO:teuthology.orchestra.run.smithi098.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-25T20:53:49.465 INFO:teuthology.orchestra.run.smithi098.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T20:53:49.490 INFO:teuthology.orchestra.run.smithi098.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-25T20:53:49.516 INFO:teuthology.orchestra.run.smithi098.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T20:53:49.541 INFO:teuthology.orchestra.run.smithi098.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T20:53:49.566 INFO:teuthology.orchestra.run.smithi098.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T20:53:49.666 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.5 MB/s | 3.5 MB 00:00 2024-04-25T20:53:49.708 INFO:teuthology.orchestra.run.smithi098.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-04-25T20:53:49.734 INFO:teuthology.orchestra.run.smithi098.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-04-25T20:53:49.759 INFO:teuthology.orchestra.run.smithi098.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-25T20:53:49.784 INFO:teuthology.orchestra.run.smithi098.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T20:53:49.809 INFO:teuthology.orchestra.run.smithi098.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T20:53:49.835 INFO:teuthology.orchestra.run.smithi098.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T20:53:49.860 INFO:teuthology.orchestra.run.smithi098.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T20:53:49.886 INFO:teuthology.orchestra.run.smithi098.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T20:53:49.911 INFO:teuthology.orchestra.run.smithi098.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T20:53:49.936 INFO:teuthology.orchestra.run.smithi098.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T20:53:50.112 INFO:teuthology.orchestra.run.smithi098.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 240 kB/s | 42 kB 00:00 2024-04-25T20:53:50.345 INFO:teuthology.orchestra.run.smithi098.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T20:53:50.370 INFO:teuthology.orchestra.run.smithi098.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 172 kB/s | 44 kB 00:00 2024-04-25T20:53:50.395 INFO:teuthology.orchestra.run.smithi098.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T20:53:50.425 INFO:teuthology.orchestra.run.smithi098.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 666 kB/s | 36 kB 00:00 2024-04-25T20:53:50.446 INFO:teuthology.orchestra.run.smithi098.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-04-25T20:53:50.496 INFO:teuthology.orchestra.run.smithi098.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.2 MB/s | 305 kB 00:00 2024-04-25T20:53:50.530 INFO:teuthology.orchestra.run.smithi098.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 690 kB/s | 58 kB 00:00 2024-04-25T20:53:50.556 INFO:teuthology.orchestra.run.smithi098.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T20:53:50.581 INFO:teuthology.orchestra.run.smithi098.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T20:53:50.599 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 7.9 MB/s | 7.4 MB 00:00 2024-04-25T20:53:50.606 INFO:teuthology.orchestra.run.smithi098.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-25T20:53:50.631 INFO:teuthology.orchestra.run.smithi098.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T20:53:50.674 INFO:teuthology.orchestra.run.smithi098.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T20:53:50.691 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.7 MB/s | 249 kB 00:00 2024-04-25T20:53:50.774 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 595 kB/s | 49 kB 00:00 2024-04-25T20:53:50.917 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.9 MB/s | 11 MB 00:05 2024-04-25T20:53:50.942 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 98 kB/s | 16 kB 00:00 2024-04-25T20:53:51.067 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 273 kB 00:00 2024-04-25T20:53:51.092 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.0 MB/s | 762 kB 00:00 2024-04-25T20:53:51.107 INFO:teuthology.orchestra.run.smithi098.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 713 kB/s | 309 kB 00:00 2024-04-25T20:53:51.143 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 243 kB/s | 18 kB 00:00 2024-04-25T20:53:51.207 INFO:teuthology.orchestra.run.smithi098.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-25T20:53:51.233 INFO:teuthology.orchestra.run.smithi098.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 914 kB/s | 548 kB 00:00 2024-04-25T20:53:51.293 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-04-25T20:53:51.327 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.8 MB/s | 195 kB 00:00 2024-04-25T20:53:51.333 INFO:teuthology.orchestra.run.smithi098.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-25T20:53:51.360 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 7.5 MB/s | 253 kB 00:00 2024-04-25T20:53:51.427 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:53:51.439 INFO:teuthology.orchestra.run.smithi177.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.9 MB/s | 11 MB 00:03 2024-04-25T20:53:51.441 INFO:teuthology.orchestra.run.smithi098.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-25T20:53:51.477 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T20:53:51.536 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T20:53:51.550 INFO:teuthology.orchestra.run.smithi098.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-04-25T20:53:51.564 INFO:teuthology.orchestra.run.smithi177.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.8 MB/s | 225 kB 00:00 2024-04-25T20:53:51.570 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T20:53:51.603 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 6.3 MB/s | 215 kB 00:00 2024-04-25T20:53:51.625 INFO:teuthology.orchestra.run.smithi098.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T20:53:51.631 INFO:teuthology.orchestra.run.smithi177.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 406 kB/s | 27 kB 00:00 2024-04-25T20:53:51.650 INFO:teuthology.orchestra.run.smithi098.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T20:53:51.664 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 11 MB/s | 689 kB 00:00 2024-04-25T20:53:51.715 INFO:teuthology.orchestra.run.smithi177.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 380 kB/s | 32 kB 00:00 2024-04-25T20:53:51.768 INFO:teuthology.orchestra.run.smithi098.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-25T20:53:51.773 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-25T20:53:51.806 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.2 MB/s | 107 kB 00:00 2024-04-25T20:53:51.831 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 867 kB/s | 21 kB 00:00 2024-04-25T20:53:51.859 INFO:teuthology.orchestra.run.smithi098.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 475 kB/s | 43 kB 00:00 2024-04-25T20:53:51.873 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T20:53:51.893 INFO:teuthology.orchestra.run.smithi098.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-25T20:53:51.899 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 33 kB 00:00 2024-04-25T20:53:51.923 INFO:teuthology.orchestra.run.smithi177.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.5 MB/s | 747 kB 00:00 2024-04-25T20:53:51.968 INFO:teuthology.orchestra.run.smithi098.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-25T20:53:51.991 INFO:teuthology.orchestra.run.smithi177.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 163 kB 00:00 2024-04-25T20:53:51.991 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-25T20:53:51.993 INFO:teuthology.orchestra.run.smithi098.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-25T20:53:52.025 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 547 kB/s | 18 kB 00:00 2024-04-25T20:53:52.058 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 337 kB/s | 11 kB 00:00 2024-04-25T20:53:52.066 INFO:teuthology.orchestra.run.smithi177.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.7 MB/s | 126 kB 00:00 2024-04-25T20:53:52.069 INFO:teuthology.orchestra.run.smithi098.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T20:53:52.084 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.3 MB/s | 160 kB 00:00 2024-04-25T20:53:52.094 INFO:teuthology.orchestra.run.smithi098.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-25T20:53:52.117 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T20:53:52.168 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T20:53:52.170 INFO:teuthology.orchestra.run.smithi098.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-04-25T20:53:52.195 INFO:teuthology.orchestra.run.smithi098.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-25T20:53:52.217 INFO:teuthology.orchestra.run.smithi177.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.3 MB/s | 500 kB 00:00 2024-04-25T20:53:52.217 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 582 kB 00:00 2024-04-25T20:53:52.251 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 765 kB/s | 25 kB 00:00 2024-04-25T20:53:52.270 INFO:teuthology.orchestra.run.smithi098.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T20:53:52.285 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T20:53:52.326 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T20:53:52.360 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 188 kB 00:00 2024-04-25T20:53:52.370 INFO:teuthology.orchestra.run.smithi098.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-25T20:53:52.394 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-25T20:53:52.471 INFO:teuthology.orchestra.run.smithi098.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-04-25T20:53:52.488 INFO:teuthology.orchestra.run.smithi098.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-25T20:53:52.511 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-25T20:53:52.579 INFO:teuthology.orchestra.run.smithi098.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T20:53:52.596 INFO:teuthology.orchestra.run.smithi098.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T20:53:52.619 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-04-25T20:53:52.672 INFO:teuthology.orchestra.run.smithi098.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T20:53:52.678 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-04-25T20:53:52.697 INFO:teuthology.orchestra.run.smithi098.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-25T20:53:52.728 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-04-25T20:53:52.754 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 5.2 MB/s | 133 kB 00:00 2024-04-25T20:53:52.789 INFO:teuthology.orchestra.run.smithi098.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-04-25T20:53:52.881 INFO:teuthology.orchestra.run.smithi098.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 861 kB/s | 79 kB 00:00 2024-04-25T20:53:52.895 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 41 MB/s | 5.8 MB 00:00 2024-04-25T20:53:52.923 INFO:teuthology.orchestra.run.smithi098.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 4.1 MB/s | 1.0 MB 00:00 2024-04-25T20:53:52.946 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-25T20:53:52.980 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-04-25T20:53:53.015 INFO:teuthology.orchestra.run.smithi098.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 433 kB/s | 58 kB 00:00 2024-04-25T20:53:53.040 INFO:teuthology.orchestra.run.smithi098.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T20:53:53.107 INFO:teuthology.orchestra.run.smithi098.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-25T20:53:53.122 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-04-25T20:53:53.132 INFO:teuthology.orchestra.run.smithi098.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-04-25T20:53:53.141 INFO:teuthology.orchestra.run.smithi177.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.0 MB/s | 17 MB 00:05 2024-04-25T20:53:53.155 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 7.2 MB/s | 244 kB 00:00 2024-04-25T20:53:53.192 INFO:teuthology.orchestra.run.smithi177.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 907 kB/s | 45 kB 00:00 2024-04-25T20:53:53.199 INFO:teuthology.orchestra.run.smithi098.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-25T20:53:53.214 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-25T20:53:53.225 INFO:teuthology.orchestra.run.smithi098.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-25T20:53:53.242 INFO:teuthology.orchestra.run.smithi177.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.6 MB/s | 135 kB 00:00 2024-04-25T20:53:53.281 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): fuse-2.9.9-15.el9.x86_64.rpm 37 kB/s | 80 kB 00:02 2024-04-25T20:53:53.292 INFO:teuthology.orchestra.run.smithi098.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-25T20:53:53.298 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 310 kB/s | 26 kB 00:00 2024-04-25T20:53:53.309 INFO:teuthology.orchestra.run.smithi177.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T20:53:53.315 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-04-25T20:53:53.317 INFO:teuthology.orchestra.run.smithi098.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-25T20:53:53.348 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T20:53:53.365 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-25T20:53:53.384 INFO:teuthology.orchestra.run.smithi098.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-04-25T20:53:53.409 INFO:teuthology.orchestra.run.smithi098.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-25T20:53:53.410 INFO:teuthology.orchestra.run.smithi177.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.1 MB/s | 320 kB 00:00 2024-04-25T20:53:53.424 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T20:53:53.476 INFO:teuthology.orchestra.run.smithi098.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T20:53:53.499 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-25T20:53:53.510 INFO:teuthology.orchestra.run.smithi177.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-25T20:53:53.524 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T20:53:53.526 INFO:teuthology.orchestra.run.smithi098.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 768 kB/s | 90 kB 00:00 2024-04-25T20:53:53.550 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T20:53:53.561 INFO:teuthology.orchestra.run.smithi177.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-25T20:53:53.575 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-25T20:53:53.585 INFO:teuthology.orchestra.run.smithi098.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T20:53:53.600 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T20:53:53.618 INFO:teuthology.orchestra.run.smithi098.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-25T20:53:53.626 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T20:53:53.635 INFO:teuthology.orchestra.run.smithi177.stdout:(25/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 5.4 MB 00:01 2024-04-25T20:53:53.651 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T20:53:53.661 INFO:teuthology.orchestra.run.smithi177.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 851 kB/s | 85 kB 00:00 2024-04-25T20:53:53.668 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T20:53:53.677 INFO:teuthology.orchestra.run.smithi098.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T20:53:53.693 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-25T20:53:53.769 INFO:teuthology.orchestra.run.smithi177.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T20:53:53.811 INFO:teuthology.orchestra.run.smithi177.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 821 kB/s | 34 kB 00:00 2024-04-25T20:53:53.818 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 339 kB/s | 42 kB 00:00 2024-04-25T20:53:53.862 INFO:teuthology.orchestra.run.smithi177.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-25T20:53:53.977 INFO:teuthology.orchestra.run.smithi098.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-04-25T20:53:54.011 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-04-25T20:53:54.036 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 206 kB/s | 44 kB 00:00 2024-04-25T20:53:54.061 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T20:53:54.078 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-04-25T20:53:54.095 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T20:53:54.120 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 305 kB 00:00 2024-04-25T20:53:54.137 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-04-25T20:53:54.162 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T20:53:54.179 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T20:53:54.196 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T20:53:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 668 kB/s | 22 kB 00:00 2024-04-25T20:53:54.255 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T20:53:54.444 INFO:teuthology.orchestra.run.smithi098.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.4 MB/s | 3.4 MB 00:00 2024-04-25T20:53:54.520 INFO:teuthology.orchestra.run.smithi098.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T20:53:54.571 INFO:teuthology.orchestra.run.smithi177.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.3 MB/s | 3.1 MB 00:00 2024-04-25T20:53:54.603 INFO:teuthology.orchestra.run.smithi098.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-25T20:53:54.653 INFO:teuthology.orchestra.run.smithi098.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.7 MB/s | 3.1 MB 00:00 2024-04-25T20:53:54.689 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 5.9 MB/s | 49 MB 00:08 2024-04-25T20:53:54.796 INFO:teuthology.orchestra.run.smithi177.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.8 MB/s | 3.5 MB 00:00 2024-04-25T20:53:54.871 INFO:teuthology.orchestra.run.smithi177.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.2 MB/s | 249 kB 00:00 2024-04-25T20:53:54.921 INFO:teuthology.orchestra.run.smithi177.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 989 kB/s | 49 kB 00:00 2024-04-25T20:53:54.923 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 83 kB/s | 19 kB 00:00 2024-04-25T20:53:55.065 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 382 kB/s | 309 kB 00:00 2024-04-25T20:53:55.090 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 626 kB/s | 548 kB 00:00 2024-04-25T20:53:55.123 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-25T20:53:55.157 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-04-25T20:53:55.207 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-04-25T20:53:55.233 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T20:53:55.347 INFO:teuthology.orchestra.run.smithi177.stdout:(34/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 6.9 MB/s | 49 MB 00:07 2024-04-25T20:53:55.372 INFO:teuthology.orchestra.run.smithi177.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 37 kB/s | 16 kB 00:00 2024-04-25T20:53:55.408 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-25T20:53:55.413 INFO:teuthology.orchestra.run.smithi177.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.0 MB/s | 273 kB 00:00 2024-04-25T20:53:55.433 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-04-25T20:53:55.456 INFO:teuthology.orchestra.run.smithi177.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 439 kB/s | 18 kB 00:00 2024-04-25T20:53:55.492 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-25T20:53:55.517 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-04-25T20:53:55.531 INFO:teuthology.orchestra.run.smithi177.stdout:(38/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.7 MB/s | 762 kB 00:00 2024-04-25T20:53:55.584 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-04-25T20:53:55.589 INFO:teuthology.orchestra.run.smithi177.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-04-25T20:53:55.609 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-25T20:53:55.640 INFO:teuthology.orchestra.run.smithi177.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-25T20:53:55.643 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T20:53:55.676 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T20:53:55.702 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T20:53:55.727 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-25T20:53:55.749 INFO:teuthology.orchestra.run.smithi177.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-25T20:53:55.761 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T20:53:55.782 INFO:teuthology.orchestra.run.smithi177.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-25T20:53:55.824 INFO:teuthology.orchestra.run.smithi177.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.2 MB 00:00 2024-04-25T20:53:55.850 INFO:teuthology.orchestra.run.smithi177.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T20:53:55.875 INFO:teuthology.orchestra.run.smithi177.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T20:53:55.900 INFO:teuthology.orchestra.run.smithi177.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T20:53:55.925 INFO:teuthology.orchestra.run.smithi177.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T20:53:55.944 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-04-25T20:53:55.959 INFO:teuthology.orchestra.run.smithi177.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T20:53:55.970 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 51 kB/s | 11 kB 00:00 2024-04-25T20:53:55.987 INFO:teuthology.orchestra.run.smithi098.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.1 MB/s | 49 MB 00:11 2024-04-25T20:53:55.993 INFO:teuthology.orchestra.run.smithi177.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T20:53:55.993 INFO:teuthology.orchestra.run.smithi098.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:53:55.993 INFO:teuthology.orchestra.run.smithi098.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-25T20:53:55.995 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 541 kB/s | 144 kB 00:00 2024-04-25T20:53:56.018 INFO:teuthology.orchestra.run.smithi177.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T20:53:56.020 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-25T20:53:56.043 INFO:teuthology.orchestra.run.smithi177.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T20:53:56.045 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-04-25T20:53:56.069 INFO:teuthology.orchestra.run.smithi177.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T20:53:56.071 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T20:53:56.094 INFO:teuthology.orchestra.run.smithi177.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T20:53:56.096 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-25T20:53:56.169 INFO:teuthology.orchestra.run.smithi177.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T20:53:56.194 INFO:teuthology.orchestra.run.smithi177.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 187 kB/s | 18 kB 00:00 2024-04-25T20:53:56.220 INFO:teuthology.orchestra.run.smithi177.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-25T20:53:56.245 INFO:teuthology.orchestra.run.smithi177.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T20:53:56.263 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-25T20:53:56.270 INFO:teuthology.orchestra.run.smithi177.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T20:53:56.288 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 213 kB/s | 46 kB 00:00 2024-04-25T20:53:56.296 INFO:teuthology.orchestra.run.smithi177.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T20:53:56.314 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 363 kB/s | 79 kB 00:00 2024-04-25T20:53:56.329 INFO:teuthology.orchestra.run.smithi177.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T20:53:56.339 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2024-04-25T20:53:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 519 kB/s | 25 kB 00:00 2024-04-25T20:53:56.364 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T20:53:56.390 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-25T20:53:56.415 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T20:53:56.430 INFO:teuthology.orchestra.run.smithi177.stdout:(62/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-25T20:53:56.440 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T20:53:56.447 INFO:teuthology.orchestra.run.smithi177.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-04-25T20:53:56.465 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T20:53:56.472 INFO:teuthology.orchestra.run.smithi177.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-25T20:53:56.497 INFO:teuthology.orchestra.run.smithi177.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T20:53:56.514 INFO:teuthology.orchestra.run.smithi177.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 853 kB/s | 35 kB 00:00 2024-04-25T20:53:56.524 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T20:53:56.549 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-25T20:53:56.574 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 441 kB/s | 59 kB 00:00 2024-04-25T20:53:56.608 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-25T20:53:56.633 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T20:53:56.683 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-04-25T20:53:56.709 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-25T20:53:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T20:53:56.776 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T20:53:56.856 INFO:teuthology.orchestra.run.smithi177.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-25T20:53:56.918 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T20:53:56.985 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T20:53:56.990 INFO:teuthology.orchestra.run.smithi177.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-04-25T20:53:57.032 INFO:teuthology.orchestra.run.smithi177.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-04-25T20:53:57.057 INFO:teuthology.orchestra.run.smithi177.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.5 MB/s | 911 kB 00:00 2024-04-25T20:53:57.068 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-25T20:53:57.082 INFO:teuthology.orchestra.run.smithi177.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.5 MB/s | 133 kB 00:00 2024-04-25T20:53:57.093 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction check 2024-04-25T20:53:57.174 INFO:teuthology.orchestra.run.smithi177.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-25T20:53:57.219 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.0 MB/s | 3.4 MB 00:00 2024-04-25T20:53:57.235 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction check succeeded. 2024-04-25T20:53:57.235 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction test 2024-04-25T20:53:57.257 INFO:teuthology.orchestra.run.smithi177.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T20:53:57.283 INFO:teuthology.orchestra.run.smithi177.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 439 kB/s | 87 kB 00:00 2024-04-25T20:53:57.392 INFO:teuthology.orchestra.run.smithi177.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-04-25T20:53:57.417 INFO:teuthology.orchestra.run.smithi177.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T20:53:57.442 INFO:teuthology.orchestra.run.smithi177.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-25T20:53:57.467 INFO:teuthology.orchestra.run.smithi177.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-04-25T20:53:57.493 INFO:teuthology.orchestra.run.smithi177.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T20:53:57.518 INFO:teuthology.orchestra.run.smithi177.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T20:53:57.543 INFO:teuthology.orchestra.run.smithi177.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-04-25T20:53:57.710 INFO:teuthology.orchestra.run.smithi177.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T20:53:57.735 INFO:teuthology.orchestra.run.smithi177.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-25T20:53:57.761 INFO:teuthology.orchestra.run.smithi177.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-04-25T20:53:57.786 INFO:teuthology.orchestra.run.smithi177.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T20:53:57.811 INFO:teuthology.orchestra.run.smithi177.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-04-25T20:53:57.836 INFO:teuthology.orchestra.run.smithi177.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T20:53:57.862 INFO:teuthology.orchestra.run.smithi177.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T20:53:57.887 INFO:teuthology.orchestra.run.smithi177.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T20:53:57.894 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 3.1 MB 00:01 2024-04-25T20:53:57.900 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:53:57.900 INFO:teuthology.orchestra.run.smithi037.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-25T20:53:57.912 INFO:teuthology.orchestra.run.smithi177.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 681 kB/s | 51 kB 00:00 2024-04-25T20:53:57.937 INFO:teuthology.orchestra.run.smithi177.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 717 kB/s | 54 kB 00:00 2024-04-25T20:53:58.087 INFO:teuthology.orchestra.run.smithi177.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 238 kB/s | 42 kB 00:00 2024-04-25T20:53:58.263 INFO:teuthology.orchestra.run.smithi177.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 136 kB/s | 44 kB 00:00 2024-04-25T20:53:58.363 INFO:teuthology.orchestra.run.smithi177.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 806 kB/s | 221 kB 00:00 2024-04-25T20:53:58.605 INFO:teuthology.orchestra.run.smithi177.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T20:53:58.630 INFO:teuthology.orchestra.run.smithi177.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 98 kB/s | 36 kB 00:00 2024-04-25T20:53:58.655 INFO:teuthology.orchestra.run.smithi177.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 163 kB/s | 47 kB 00:00 2024-04-25T20:53:58.681 INFO:teuthology.orchestra.run.smithi177.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T20:53:58.706 INFO:teuthology.orchestra.run.smithi177.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T20:53:58.731 INFO:teuthology.orchestra.run.smithi177.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T20:53:58.765 INFO:teuthology.orchestra.run.smithi177.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-25T20:53:58.790 INFO:teuthology.orchestra.run.smithi177.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-25T20:53:58.815 INFO:teuthology.orchestra.run.smithi177.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-04-25T20:53:58.857 INFO:teuthology.orchestra.run.smithi177.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T20:53:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:Transaction test succeeded. 2024-04-25T20:53:59.007 INFO:teuthology.orchestra.run.smithi098.stdout:Running transaction 2024-04-25T20:53:59.043 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-25T20:53:59.116 INFO:teuthology.orchestra.run.smithi177.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-25T20:53:59.205 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-25T20:53:59.205 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-25T20:53:59.450 INFO:teuthology.orchestra.run.smithi177.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 488 kB/s | 309 kB 00:00 2024-04-25T20:53:59.566 INFO:teuthology.orchestra.run.smithi177.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 731 kB/s | 548 kB 00:00 2024-04-25T20:53:59.709 INFO:teuthology.orchestra.run.smithi177.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-25T20:53:59.909 INFO:teuthology.orchestra.run.smithi177.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T20:54:00.051 INFO:teuthology.orchestra.run.smithi177.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-25T20:54:00.134 INFO:teuthology.orchestra.run.smithi177.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-25T20:54:00.343 INFO:teuthology.orchestra.run.smithi177.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 348 kB/s | 151 kB 00:00 2024-04-25T20:54:00.685 INFO:teuthology.orchestra.run.smithi177.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-25T20:54:00.718 INFO:teuthology.orchestra.run.smithi177.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T20:54:00.743 INFO:teuthology.orchestra.run.smithi177.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 900 kB/s | 548 kB 00:00 2024-04-25T20:54:00.960 INFO:teuthology.orchestra.run.smithi177.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-04-25T20:54:00.986 INFO:teuthology.orchestra.run.smithi177.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-25T20:54:01.076 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-25T20:54:01.078 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-25T20:54:01.261 INFO:teuthology.orchestra.run.smithi177.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 332 kB/s | 172 kB 00:00 2024-04-25T20:54:01.470 INFO:teuthology.orchestra.run.smithi177.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 704 kB/s | 358 kB 00:00 2024-04-25T20:54:01.528 INFO:teuthology.orchestra.run.smithi177.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-25T20:54:01.571 INFO:teuthology.orchestra.run.smithi177.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 519 kB/s | 303 kB 00:00 2024-04-25T20:54:02.004 INFO:teuthology.orchestra.run.smithi177.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 333 kB/s | 144 kB 00:00 2024-04-25T20:54:02.029 INFO:teuthology.orchestra.run.smithi177.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 368 kB/s | 206 kB 00:00 2024-04-25T20:54:02.196 INFO:teuthology.orchestra.run.smithi177.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-25T20:54:02.288 INFO:teuthology.orchestra.run.smithi177.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-04-25T20:54:02.413 INFO:teuthology.orchestra.run.smithi177.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-25T20:54:02.472 INFO:teuthology.orchestra.run.smithi177.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 84 kB/s | 23 kB 00:00 2024-04-25T20:54:02.547 INFO:teuthology.orchestra.run.smithi177.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 75 kB/s | 19 kB 00:00 2024-04-25T20:54:02.681 INFO:teuthology.orchestra.run.smithi177.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-25T20:54:02.889 INFO:teuthology.orchestra.run.smithi177.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-25T20:54:03.023 INFO:teuthology.orchestra.run.smithi177.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 231 kB/s | 79 kB 00:00 2024-04-25T20:54:03.173 INFO:teuthology.orchestra.run.smithi177.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-04-25T20:54:03.231 INFO:teuthology.orchestra.run.smithi177.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-25T20:54:03.440 INFO:teuthology.orchestra.run.smithi177.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-25T20:54:03.532 INFO:teuthology.orchestra.run.smithi177.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-25T20:54:03.582 INFO:teuthology.orchestra.run.smithi177.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 256 kB/s | 90 kB 00:00 2024-04-25T20:54:03.725 INFO:teuthology.orchestra.run.smithi177.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 99 kB/s | 28 kB 00:00 2024-04-25T20:54:03.792 INFO:teuthology.orchestra.run.smithi177.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-04-25T20:54:04.008 INFO:teuthology.orchestra.run.smithi177.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 443 kB/s | 188 kB 00:00 2024-04-25T20:54:04.059 INFO:teuthology.orchestra.run.smithi177.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-25T20:54:04.084 INFO:teuthology.orchestra.run.smithi177.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 165 kB/s | 59 kB 00:00 2024-04-25T20:54:04.359 INFO:teuthology.orchestra.run.smithi177.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-04-25T20:54:04.426 INFO:teuthology.orchestra.run.smithi177.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-04-25T20:54:04.568 INFO:teuthology.orchestra.run.smithi177.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-04-25T20:54:04.693 INFO:teuthology.orchestra.run.smithi177.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-25T20:54:04.943 INFO:teuthology.orchestra.run.smithi177.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-04-25T20:54:04.994 INFO:teuthology.orchestra.run.smithi177.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-04-25T20:54:05.134 INFO:teuthology.orchestra.run.smithi098.stdout: Preparing : 1/1 2024-04-25T20:54:05.208 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:54:05.246 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:54:05.300 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:54:05.333 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:54:05.359 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:54:05.401 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:54:05.452 INFO:teuthology.orchestra.run.smithi177.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T20:54:05.536 INFO:teuthology.orchestra.run.smithi177.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T20:54:05.661 INFO:teuthology.orchestra.run.smithi177.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T20:54:05.724 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:54:05.835 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:05.957 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:05.986 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:06.061 INFO:teuthology.orchestra.run.smithi177.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.0 MB/s | 3.4 MB 00:01 2024-04-25T20:54:06.077 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:06.115 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:54:06.186 INFO:teuthology.orchestra.run.smithi177.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-25T20:54:06.192 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:54:06.192 INFO:teuthology.orchestra.run.smithi177.stdout:Total 8.6 MB/s | 222 MB 00:25 2024-04-25T20:54:06.264 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:54:06.312 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:54:06.341 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:06.443 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:06.466 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:06.529 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:06.551 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:06.568 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:06.725 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:06.759 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:06.865 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:54:06.963 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:54:07.008 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:54:07.043 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:54:07.117 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:54:07.151 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:54:07.195 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:54:07.227 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:54:07.249 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-25T20:54:07.255 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:54:07.276 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2024-04-25T20:54:07.325 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:54:07.348 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:54:07.394 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:54:07.395 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:54:07.418 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2024-04-25T20:54:07.418 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2024-04-25T20:54:07.424 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:54:07.449 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:54:07.468 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:54:07.491 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:54:07.516 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:54:07.546 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:54:07.560 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:54:07.579 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:54:07.622 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:54:07.651 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:54:07.720 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:54:07.766 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:54:07.840 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:54:07.956 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:54:07.975 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:54:08.017 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:08.026 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:54:08.124 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:08.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:08.252 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:08.281 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:54:08.443 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:54:08.495 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:54:08.522 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:08.559 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:54:08.628 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:08.637 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:54:08.657 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:08.667 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:54:08.710 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:54:08.720 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:08.741 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:08.748 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:54:08.766 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:08.789 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:54:08.917 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:08.949 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:08.962 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:54:09.018 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:54:09.061 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:54:09.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:54:09.136 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:54:09.159 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:54:09.168 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:54:09.199 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:54:09.205 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:54:09.234 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:54:09.240 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2024-04-25T20:54:09.240 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2024-04-25T20:54:09.260 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:54:09.296 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:54:09.311 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:54:09.339 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:54:09.350 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:54:09.381 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:54:09.394 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:54:09.426 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:54:09.462 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:54:09.515 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:54:09.535 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:54:09.562 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:54:09.586 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:54:09.592 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:54:09.622 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:54:09.633 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:54:09.658 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:54:09.726 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:54:09.738 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:54:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:54:09.769 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:54:09.799 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:54:09.813 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:54:09.825 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:54:09.841 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:54:09.849 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:54:09.916 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:54:09.924 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:54:09.973 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:54:10.049 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:54:10.056 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:54:10.082 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:54:10.109 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:54:10.144 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:54:10.183 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:54:10.200 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:54:10.223 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:54:10.253 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:54:10.261 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:54:10.287 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:54:10.321 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:54:10.362 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:54:10.489 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:54:10.725 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:54:10.770 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:54:10.800 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:54:10.836 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:54:10.848 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:54:10.925 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:54:10.958 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:54:10.964 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:54:11.001 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:54:11.018 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:54:11.039 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:54:11.082 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:54:11.251 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:54:11.301 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:54:11.349 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:54:11.424 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:54:11.459 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:54:11.496 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:54:11.545 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:54:11.579 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:54:11.623 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:54:11.665 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:54:11.739 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:54:11.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:54:11.811 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:54:11.824 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:54:11.853 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:54:11.883 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:54:11.922 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:54:11.925 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:54:12.007 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:54:12.025 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:54:12.058 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:54:12.096 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:54:12.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:54:12.132 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:54:12.134 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:54:12.156 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:54:12.247 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:54:12.312 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:54:12.349 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:54:12.382 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:54:12.422 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:54:12.427 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:54:12.450 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:54:12.484 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:54:12.524 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:54:12.564 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:54:12.593 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:54:12.628 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:54:12.661 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:54:12.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:54:12.804 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:54:12.896 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:54:12.952 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:54:12.982 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:54:13.031 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:54:13.094 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:54:13.151 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:54:13.202 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:54:13.206 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:54:13.989 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:54:14.040 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:54:14.073 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:54:14.203 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:54:14.266 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:54:14.358 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:54:14.392 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:54:14.429 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:54:14.504 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:54:14.532 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:54:14.565 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:54:14.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:54:14.606 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:54:14.678 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:54:14.878 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:54:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:15.010 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:15.041 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:54:15.077 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:54:15.272 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2024-04-25T20:54:15.348 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:54:15.376 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:54:15.395 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:54:15.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:54:15.493 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:54:15.501 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:54:15.518 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:54:15.557 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:54:15.561 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:54:15.598 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:15.653 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:15.683 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:54:15.887 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:54:15.994 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:16.116 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:16.145 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:16.237 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:16.266 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:54:16.415 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:54:16.463 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:54:16.491 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:16.578 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:16.601 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:16.655 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:54:16.677 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:16.701 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:54:16.841 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:54:16.893 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:16.918 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:54:16.946 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:54:16.952 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:54:16.980 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:54:17.058 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:54:17.147 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:54:17.192 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:54:17.227 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:54:17.292 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:54:17.309 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:54:17.326 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:54:17.350 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:17.369 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:54:17.402 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:54:17.430 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:54:17.442 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:17.472 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:54:17.499 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:54:17.508 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:54:17.552 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:54:17.583 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:54:17.628 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:54:17.702 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:54:17.737 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:54:17.780 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:54:17.809 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:54:17.880 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:54:17.932 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:54:18.007 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:54:18.007 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:18.015 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:54:18.045 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:18.046 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:18.116 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:18.144 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:54:18.146 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:54:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:54:18.593 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:18.634 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:18.741 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:18.742 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:54:18.742 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:54:18.742 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:18.758 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:54:18.760 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:18.840 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:54:18.869 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:54:18.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:54:18.950 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:54:18.992 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:54:19.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:54:19.211 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:54:19.266 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:54:19.339 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:54:19.370 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:54:19.406 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:54:19.453 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:54:19.489 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:54:19.532 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:54:19.575 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:54:19.712 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:54:19.756 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:54:19.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:54:19.827 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:54:19.922 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:54:19.952 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:54:19.992 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:54:20.018 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:54:20.042 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:54:20.117 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:54:20.245 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:54:20.275 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:54:20.303 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:54:20.337 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:54:20.377 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:54:20.416 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:54:20.446 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:54:20.481 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:54:20.523 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:54:20.549 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:20.564 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:54:20.585 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:20.599 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:54:20.686 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:54:20.729 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:54:20.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:54:20.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:54:21.148 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:54:21.159 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:21.195 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:54:21.205 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:21.288 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:21.288 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:54:21.288 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:54:21.288 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:21.323 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:21.928 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:54:21.967 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:54:22.001 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:54:22.102 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:54:22.159 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:54:22.247 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:54:22.278 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:54:22.463 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:54:22.492 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:54:22.570 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:54:23.222 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:54:23.342 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:54:24.581 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:54:24.664 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:54:24.692 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:54:24.725 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:54:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:54:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:25.162 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:54:25.192 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:54:25.245 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:54:25.724 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:54:25.757 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:25.828 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:54:25.851 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:54:28.144 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:28.181 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:28.727 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:54:28.773 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:54:28.858 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:54:28.858 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:54:28.858 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:28.885 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:30.355 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:30.355 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /sys 2024-04-25T20:54:30.355 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /proc 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /mnt 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /var/tmp 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /home 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /root 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout:skipping the directory /tmp 2024-04-25T20:54:30.356 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:30.549 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:30.564 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:30.565 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:30.565 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:54:30.565 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:32.028 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:32.054 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:32.054 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:32.054 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:54:32.054 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:32.054 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:32.055 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:32.278 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:32.300 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:32.300 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:32.300 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:54:32.300 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:32.301 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:32.301 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:32.737 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:32.759 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:32.760 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:32.760 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:54:32.760 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:32.760 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:32.760 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:32.868 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:54:32.925 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:54:32.981 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-25T20:54:33.070 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-25T20:54:33.071 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:33.166 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:54:33.213 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:54:33.244 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:33.260 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:33.260 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:33.260 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:54:33.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:33.282 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:54:33.372 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:54:33.510 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:54:33.555 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:54:33.587 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:54:33.615 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:54:34.041 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:34.072 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:34.112 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:54:34.174 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:54:34.204 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:35.088 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:35.088 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:54:35.088 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:35.089 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:35.233 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:35.264 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:54:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:54:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:54:36.007 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:54:36.058 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:54:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:54:36.220 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:36.237 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:54:36.256 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:36.356 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:36.383 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:54:36.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:54:36.457 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:54:36.484 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:54:36.518 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:54:36.541 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:36.564 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:36.609 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:36.620 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:36.717 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:54:36.766 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:54:36.965 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:36.983 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:37.016 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:54:37.080 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:54:37.105 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:37.692 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:54:37.720 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:37.742 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:54:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:37.743 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:37.787 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:37.809 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:37.809 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:37.809 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:54:37.809 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:37.854 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:37.877 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:38.172 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:38.194 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:38.210 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:38.249 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:39.270 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:39.299 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:39.411 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:39.579 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:54:39.610 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:39.635 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:39.686 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:39.697 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:39.816 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:54:39.869 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /root 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2024-04-25T20:54:40.214 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:40.390 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:40.408 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:54:40.408 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:40.408 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:54:40.408 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:40.848 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:54:40.871 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:40.895 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:40.895 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:40.896 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:54:40.896 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:40.896 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:40.896 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:40.939 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:40.962 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:40.962 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:40.962 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:54:40.962 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:41.014 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:41.039 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:41.365 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:41.389 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:41.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:41.918 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:54:41.918 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:41.919 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:54:41.919 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:41.919 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:54:41.919 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:42.144 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:54:42.166 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:42.603 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:54:42.625 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:42.733 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:54:42.775 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:54:42.790 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:54:42.816 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:54:42.845 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:54:42.857 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:54:42.890 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:54:42.917 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:54:43.032 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:54:43.079 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:54:43.101 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:54:43.146 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:54:43.149 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:54:43.176 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:54:43.246 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:54:43.387 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:54:43.420 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:54:43.452 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:54:43.480 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:54:43.748 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:43.791 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:43.810 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:43.925 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:43.954 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:54:43.986 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:54:44.048 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:54:44.102 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:44.654 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:54:44.740 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:54:44.781 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:54:44.862 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:54:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:54:44.903 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:44.987 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:45.020 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:54:45.020 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:45.111 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:45.139 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:54:45.171 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:45.177 INFO:teuthology.orchestra.run.smithi098.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:46.132 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:54:46.154 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:46.256 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:54:46.261 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:54:46.302 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:54:46.344 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:54:46.375 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:54:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:54:46.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:54:46.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:46.470 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:54:46.470 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:46.471 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:54:46.471 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:46.471 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:54:46.471 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:46.516 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:46.527 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:54:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:54:46.624 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:54:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:54:46.672 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:54:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:54:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:47.594 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:54:47.626 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:54:47.649 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:47.693 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:47.715 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:54:47.715 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:47.715 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:54:47.715 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:47.769 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:54:47.792 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:48.095 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:54:48.117 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:54:48.304 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:54:48.397 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:54:48.431 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:54:48.525 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:54:48.568 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:54:48.568 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:48.660 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:48.693 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:54:48.693 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:48.788 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:48.862 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:51.032 INFO:teuthology.orchestra.run.smithi098.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:51.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:54:51.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:54:51.032 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:54:51.033 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:54:51.034 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:54:51.035 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:54:51.036 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:54:51.037 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:54:51.038 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:54:51.039 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:54:51.040 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:54:51.041 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:54:51.042 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:54:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:54:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:54:52.707 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:54:52.748 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:54:52.789 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:54:52.821 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:54:52.848 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:54:53.036 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:54:53.086 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:54:53.116 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout:Upgraded: 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout:Installed: 2024-04-25T20:54:53.158 INFO:teuthology.orchestra.run.smithi098.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.159 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.160 INFO:teuthology.orchestra.run.smithi098.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:54:53.161 INFO:teuthology.orchestra.run.smithi098.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.162 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:54:53.163 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.164 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:54:53.165 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:54:53.166 INFO:teuthology.orchestra.run.smithi098.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.167 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:54:53.168 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:54:53.169 INFO:teuthology.orchestra.run.smithi098.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:54:53.170 INFO:teuthology.orchestra.run.smithi098.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:54:53.170 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-04-25T20:54:53.170 INFO:teuthology.orchestra.run.smithi098.stdout:Complete! 2024-04-25T20:54:53.632 DEBUG:teuthology.parallel:result is None 2024-04-25T20:54:53.690 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:53.730 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:53.751 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:54:54.575 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:54:54.669 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:54:54.710 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:54:54.728 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:54:54.729 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:54:54.730 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:54:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:54:54.732 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:54:54.733 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:54:54.734 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:54:54.735 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:54:54.736 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:54:54.737 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:54:54.738 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:54:54.739 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:54:54.740 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:54:54.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:54:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:54:54.823 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:54.899 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:54:54.932 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:54:54.932 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:55.023 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:54:55.087 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.775 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.776 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.777 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:54:56.778 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.779 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:54:56.780 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:54:56.781 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:54:56.782 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:54:56.783 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:54:56.784 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:54:56.785 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:54:56.786 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-25T20:54:57.169 DEBUG:teuthology.parallel:result is None 2024-04-25T20:55:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:55:00.756 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:55:00.757 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:55:00.758 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:55:00.759 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:55:00.760 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:55:00.761 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:55:00.762 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:55:00.763 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:55:00.764 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:55:00.765 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:55:00.766 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:55:00.767 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:55:00.768 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:55:00.769 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2024-04-25T20:55:02.794 INFO:teuthology.orchestra.run.smithi177.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.795 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:55:02.796 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:55:02.797 INFO:teuthology.orchestra.run.smithi177.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:55:02.798 INFO:teuthology.orchestra.run.smithi177.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:55:02.799 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:55:02.800 INFO:teuthology.orchestra.run.smithi177.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:55:02.801 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:55:02.802 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:55:02.803 INFO:teuthology.orchestra.run.smithi177.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:55:02.804 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:55:02.805 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:55:02.805 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:55:02.805 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:55:02.805 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:55:02.805 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2024-04-25T20:55:03.182 DEBUG:teuthology.parallel:result is None 2024-04-25T20:55:03.183 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:03.438 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:55:03.464 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.465 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.465 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:55:03.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:03.611 DEBUG:teuthology.orchestra.run.smithi098:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:55:03.636 INFO:teuthology.orchestra.run.smithi098.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.636 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.636 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:55:03.638 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:03.865 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:55:03.886 INFO:teuthology.orchestra.run.smithi177.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.886 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:55:03.886 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:55:03.888 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T20:55:03.888 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:55:03.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:55:03.920 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:55:03.920 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:55:03.951 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:55:03.951 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:55:03.976 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T20:55:03.977 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:55:03.977 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:55:04.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:55:04.074 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:55:04.074 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:55:04.103 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:55:04.170 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:55:04.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:55:04.193 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:55:04.259 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T20:55:04.259 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:55:04.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:55:04.288 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:55:04.356 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:55:04.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:55:04.384 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:55:04.451 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:55:04.451 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:55:04.474 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:55:04.537 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T20:55:04.538 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:55:04.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:55:04.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:55:04.634 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:55:04.634 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:55:04.662 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:55:04.728 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:55:04.728 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:55:04.751 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:55:04.814 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T20:55:04.920 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T20:55:04.920 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:04.921 INFO:tasks.cephadm:Cluster fsid is 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:55:04.921 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T20:55:04.921 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.98', 'mon.c': '172.21.15.177'} 2024-04-25T20:55:04.921 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-04-25T20:55:04.922 INFO:tasks.cephadm:First mgr is x 2024-04-25T20:55:04.922 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T20:55:04.922 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-25T20:55:04.951 DEBUG:teuthology.orchestra.run.smithi098:> sudo hostname $(hostname -s) 2024-04-25T20:55:04.979 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname $(hostname -s) 2024-04-25T20:55:05.003 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T20:55:05.003 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:05.141 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T20:55:05.425 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:55:05.427 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T20:55:05.427 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:06.375 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:55 /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:06.376 DEBUG:teuthology.orchestra.run.smithi098:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:07.246 INFO:teuthology.orchestra.run.smithi098.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:55 /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:07.246 DEBUG:teuthology.orchestra.run.smithi177:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:08.124 INFO:teuthology.orchestra.run.smithi177.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:55 /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:08.125 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:08.144 DEBUG:teuthology.orchestra.run.smithi098:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:08.163 DEBUG:teuthology.orchestra.run.smithi177:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:55:08.197 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T20:55:08.197 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:55:08.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:55:08.205 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:55:09.710 INFO:teuthology.orchestra.run.smithi098.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:55:09.780 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:55:09.864 INFO:teuthology.orchestra.run.smithi177.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:55:37.420 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-04-25T20:55:37.420 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:55:37.421 INFO:teuthology.orchestra.run.smithi177.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:55:37.421 INFO:teuthology.orchestra.run.smithi177.stdout: "repo_digests": [ 2024-04-25T20:55:37.421 INFO:teuthology.orchestra.run.smithi177.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:55:37.421 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2024-04-25T20:55:37.421 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout: "repo_digests": [ 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-04-25T20:55:50.409 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-25T20:56:20.857 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T20:56:20.858 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T20:56:20.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-25T20:56:20.922 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph 2024-04-25T20:56:20.954 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph 2024-04-25T20:56:20.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-25T20:56:21.012 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /etc/ceph 2024-04-25T20:56:21.040 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /etc/ceph 2024-04-25T20:56:21.069 INFO:tasks.cephadm:Writing seed config... 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T20:56:21.070 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T20:56:21.071 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T20:56:21.072 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T20:56:21.073 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T20:56:21.074 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:56:21.074 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T20:56:21.090 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1788dafa-0346-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T20:56:21.090 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T20:56:21.133 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T20:56:21.175 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T20:56:21.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:56:21.375 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:56:21.743 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '1788dafa-0346-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-04-25T20:56:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-25T20:56:21.749 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-25T20:56:21.750 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140146136268752 on /run/cephadm/1788dafa-0346-11ef-bc93-c7b262605968.lock 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140146136268752 acquired on /run/cephadm/1788dafa-0346-11ef-bc93-c7b262605968.lock 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-04-25T20:56:21.751 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-25T20:56:21.752 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-25T20:56:21.753 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T20:56:21.753 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T20:56:21.753 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:56:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:56:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T20:56:22.846 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:56:22.846 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:56:22.846 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T20:56:23.859 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-04-25T20:56:23.859 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-04-25T20:56:24.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQB4wypm5tVQCRAA8ETU9O6WIUzqT7Iqml3Liw== 2024-04-25T20:56:25.725 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQB5wypmctfIBRAAFcKcll/NJQg0r0UlzfIlcQ== 2024-04-25T20:56:26.607 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQB6wypmyB6gABAAgCDWUVrPupENt3CQ3uSudw== 2024-04-25T20:56:26.608 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-04-25T20:56:27.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:56:27.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:56:27.530 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-04-25T20:56:28.477 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-04-25T20:56:28.682 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T20:56:28.865 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T20:56:29.062 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target. 2024-04-25T20:56:29.062 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target. 2024-04-25T20:56:29.314 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a 2024-04-25T20:56:29.314 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T20:56:29.488 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target.wants/ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968@.service. 2024-04-25T20:56:29.925 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:29 smithi037 podman[28805]: 2024-04-25 20:56:29.634989455 +0000 UTC m=+0.018597785 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:56:30.104 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:56:30.104 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T20:56:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:56:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:56:30.112 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T20:56:30.113 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:56:30.113 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-04-25T20:56:30.113 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 podman[28805]: 2024-04-25 20:56:30.011961971 +0000 UTC m=+0.395570277 container init fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 podman[28805]: 2024-04-25 20:56:30.015634855 +0000 UTC m=+0.399243161 container start fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 ceph-mon[28824]: mkfs 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 ceph-mon[28824]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 bash[28805]: fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f 2024-04-25T20:56:30.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:30 smithi037 systemd[1]: Started Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: monmap epoch 1 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: last_changed 2024-04-25T20:56:26.923204+0000 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: min_mon_release 19 (squid) 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: election_strategy: 1 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: fsmap 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: mgrmap e1: no daemons active 2024-04-25T20:56:31.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:31 smithi037 ceph-mon[28824]: from='client.? 172.21.15.37:0/951438094' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T20:56:31.226 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T20:56:31.232 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.571652s) 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T20:56:31.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.543+0000 7ff530869700 1 Processor -- start 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.543+0000 7ff530869700 1 -- start start 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.543+0000 7ff530869700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.543+0000 7ff530869700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c108ed0 con 0x7ff52c1084e0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.544+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.544+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57872/0 (socket says 172.21.15.37:57872) 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.544+0000 7ff52a59c700 1 -- 172.21.15.37:0/4083051551 learned_addr learned my addr 172.21.15.37:0/4083051551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.544+0000 7ff52a59c700 1 -- 172.21.15.37:0/4083051551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c109690 con 0x7ff52c1084e0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.544+0000 7ff52a59c700 1 --2- 172.21.15.37:0/4083051551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b4f5b36c0a46aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff52959a700 1 -- 172.21.15.37:0/4083051551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c004160 con 0x7ff52c1084e0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff52959a700 1 -- 172.21.15.37:0/4083051551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c004300 con 0x7ff52c1084e0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff52959a700 1 -- 172.21.15.37:0/4083051551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c0036a0 con 0x7ff52c1084e0 2024-04-25T20:56:31.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 -- 172.21.15.37:0/4083051551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 msgr2=0x7ff52c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 --2- 172.21.15.37:0/4083051551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0098a0 tx=0x7ff51c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 -- 172.21.15.37:0/4083051551 shutdown_connections 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 --2- 172.21.15.37:0/4083051551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c108900 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 -- 172.21.15.37:0/4083051551 >> 172.21.15.37:0/4083051551 conn(0x7ff52c103ae0 msgr2=0x7ff52c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 -- 172.21.15.37:0/4083051551 shutdown_connections 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.545+0000 7ff530869700 1 -- 172.21.15.37:0/4083051551 wait complete. 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.546+0000 7ff530869700 1 Processor -- start 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.546+0000 7ff530869700 1 -- start start 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.546+0000 7ff530869700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.546+0000 7ff530869700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c109bc0 con 0x7ff52c1084e0 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.546+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff52a59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57876/0 (socket says 172.21.15.37:57876) 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff52a59c700 1 -- 172.21.15.37:0/951438094 learned_addr learned my addr 172.21.15.37:0/951438094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:31.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff52a59c700 1 -- 172.21.15.37:0/951438094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c07e0c0 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff52a59c700 1 --2- 172.21.15.37:0/951438094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000c00 tx=0x7ff51c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c003db0 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c011630 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c07c2b0 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff51c011950 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c07c7f0 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff51c01a460 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.547+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff51c01b780 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.548+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c0640a0 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.551+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff51c01b980 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.625+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff52c068a40 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.625+0000 7ff5237fe700 1 -- 172.21.15.37:0/951438094 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff51c01bb60 con 0x7ff52c1084e0 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 msgr2=0x7ff52c07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:31.236 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 --2- 172.21.15.37:0/951438094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff51c000c00 tx=0x7ff51c004750 comp rx=0 tx=0).stop 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 shutdown_connections 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 --2- 172.21.15.37:0/951438094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff52c1084e0 0x7ff52c07db80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 >> 172.21.15.37:0/951438094 conn(0x7ff52c103ae0 msgr2=0x7ff52c192840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 shutdown_connections 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:30.626+0000 7ff530869700 1 -- 172.21.15.37:0/951438094 wait complete. 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-04-25T20:56:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T20:56:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:32 smithi037 ceph-mon[28824]: from='client.? 172.21.15.37:0/3517833724' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:56:32.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:32 smithi037 ceph-mon[28824]: from='client.? 172.21.15.37:0/3517833724' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T20:56:32.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:56:32.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.664+0000 7f7c66570700 1 Processor -- start 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.664+0000 7f7c66570700 1 -- start start 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c66570700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c66570700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6007be60 con 0x7f7c60108f60 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57888/0 (socket says 172.21.15.37:57888) 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5ffff700 1 -- 172.21.15.37:0/3445585602 learned_addr learned my addr 172.21.15.37:0/3445585602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5ffff700 1 -- 172.21.15.37:0/3445585602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6010b890 con 0x7f7c60108f60 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5ffff700 1 --2- 172.21.15.37:0/3445585602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7c500098a0 tx=0x7f7c50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e46297b87838716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5effd700 1 -- 172.21.15.37:0/3445585602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c50004160 con 0x7f7c60108f60 2024-04-25T20:56:32.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5effd700 1 -- 172.21.15.37:0/3445585602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c50004300 con 0x7f7c60108f60 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.665+0000 7f7c5effd700 1 -- 172.21.15.37:0/3445585602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c500036a0 con 0x7f7c60108f60 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 -- 172.21.15.37:0/3445585602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 msgr2=0x7f7c6010b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 --2- 172.21.15.37:0/3445585602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7c500098a0 tx=0x7f7c50009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 -- 172.21.15.37:0/3445585602 shutdown_connections 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 --2- 172.21.15.37:0/3445585602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6010b350 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 -- 172.21.15.37:0/3445585602 >> 172.21.15.37:0/3445585602 conn(0x7f7c601029b0 msgr2=0x7f7c60104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 -- 172.21.15.37:0/3445585602 shutdown_connections 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.666+0000 7f7c66570700 1 -- 172.21.15.37:0/3445585602 wait complete. 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c66570700 1 Processor -- start 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c66570700 1 -- start start 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c66570700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c66570700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6010bd00 con 0x7f7c60108f60 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:32.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57892/0 (socket says 172.21.15.37:57892) 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c5ffff700 1 -- 172.21.15.37:0/3517833724 learned_addr learned my addr 172.21.15.37:0/3517833724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.667+0000 7f7c5ffff700 1 -- 172.21.15.37:0/3517833724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6019c130 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5ffff700 1 --2- 172.21.15.37:0/3517833724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000c00 tx=0x7f7c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c50003db0 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c50011630 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6019c3c0 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c50011950 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6019f0b0 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c5001a460 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.668+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c5001b780 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.669+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c600640a0 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.671+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7c5001ba20 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.741+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7c6002e1d0 con 0x7f7c60108f60 2024-04-25T20:56:32.462 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.744+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f7c5001bc00 con 0x7f7c60108f60 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.745+0000 7f7c5d7fa700 1 -- 172.21.15.37:0/3517833724 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7c5001bde0 con 0x7f7c60108f60 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 msgr2=0x7f7c6019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 --2- 172.21.15.37:0/3517833724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000c00 tx=0x7f7c50004750 comp rx=0 tx=0).stop 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 shutdown_connections 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 --2- 172.21.15.37:0/3517833724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c60108f60 0x7f7c6019bbf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 >> 172.21.15.37:0/3517833724 conn(0x7f7c601029b0 msgr2=0x7f7c60104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 shutdown_connections 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:31.746+0000 7f7c66570700 1 -- 172.21.15.37:0/3517833724 wait complete. 2024-04-25T20:56:32.463 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-04-25T20:56:33.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:33 smithi037 ceph-mon[28824]: from='client.? 172.21.15.37:0/3883943173' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:33.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.839+0000 7fec77739700 1 Processor -- start 2024-04-25T20:56:33.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.839+0000 7fec77739700 1 -- start start 2024-04-25T20:56:33.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.839+0000 7fec77739700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.839+0000 7fec77739700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec7007be80 con 0x7fec70108f40 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec754d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec754d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57904/0 (socket says 172.21.15.37:57904) 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec754d5700 1 -- 172.21.15.37:0/738274308 learned_addr learned my addr 172.21.15.37:0/738274308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec754d5700 1 -- 172.21.15.37:0/738274308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7010b870 con 0x7fec70108f40 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec754d5700 1 --2- 172.21.15.37:0/738274308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec600098a0 tx=0x7fec60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ef9267192d5cba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec67fff700 1 -- 172.21.15.37:0/738274308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60004160 con 0x7fec70108f40 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec67fff700 1 -- 172.21.15.37:0/738274308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec60004300 con 0x7fec70108f40 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.840+0000 7fec67fff700 1 -- 172.21.15.37:0/738274308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60011af0 con 0x7fec70108f40 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 -- 172.21.15.37:0/738274308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 msgr2=0x7fec7010b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 --2- 172.21.15.37:0/738274308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec600098a0 tx=0x7fec60009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 -- 172.21.15.37:0/738274308 shutdown_connections 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 --2- 172.21.15.37:0/738274308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7010b330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 -- 172.21.15.37:0/738274308 >> 172.21.15.37:0/738274308 conn(0x7fec701029b0 msgr2=0x7fec70104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 -- 172.21.15.37:0/738274308 shutdown_connections 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.841+0000 7fec77739700 1 -- 172.21.15.37:0/738274308 wait complete. 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec77739700 1 Processor -- start 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec77739700 1 -- start start 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec77739700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec77739700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec7010c830 con 0x7fec70108f40 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec754d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec754d5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57920/0 (socket says 172.21.15.37:57920) 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec754d5700 1 -- 172.21.15.37:0/3883943173 learned_addr learned my addr 172.21.15.37:0/3883943173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec754d5700 1 -- 172.21.15.37:0/3883943173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7019ebd0 con 0x7fec70108f40 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.842+0000 7fec754d5700 1 --2- 172.21.15.37:0/3883943173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec60009fb0 tx=0x7fec60011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec600036a0 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec6001a430 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7019ee60 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec60019550 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec701a1b50 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec60019b90 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.843+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec60024730 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.844+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec700640a0 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.846+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fec6001a5e0 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.916+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fec701a20a0 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec667fc700 1 -- 172.21.15.37:0/3883943173 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fec60032030 con 0x7fec70108f40 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 msgr2=0x7fec7019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec77739700 1 --2- 172.21.15.37:0/3883943173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec60009fb0 tx=0x7fec60011940 comp rx=0 tx=0).stop 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 shutdown_connections 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec77739700 1 --2- 172.21.15.37:0/3883943173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec70108f40 0x7fec7019e690 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.917+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 >> 172.21.15.37:0/3883943173 conn(0x7fec701029b0 msgr2=0x7fec701034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:33.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.918+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 shutdown_connections 2024-04-25T20:56:33.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:32.918+0000 7fec77739700 1 -- 172.21.15.37:0/3883943173 wait complete. 2024-04-25T20:56:33.433 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-04-25T20:56:33.693 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:33 smithi037 systemd[1]: Stopping Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T20:56:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a[28820]: 2024-04-25T20:56:33.691+0000 7fea538e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T20:56:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a[28820]: 2024-04-25T20:56:33.691+0000 7fea538e0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T20:56:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:33 smithi037 podman[29069]: 2024-04-25 20:56:33.928628006 +0000 UTC m=+0.334479192 container died fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:56:34.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 podman[29069]: 2024-04-25 20:56:34.25046132 +0000 UTC m=+0.656312495 container cleanup fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T20:56:34.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 bash[29069]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a 2024-04-25T20:56:34.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 podman[29082]: 2024-04-25 20:56:34.442902878 +0000 UTC m=+0.513184276 container remove fd7dad72b49f7234dbae42345a277da1d8614bc5aa667210dc3b992e22167d8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T20:56:34.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T20:56:34.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 systemd[1]: Stopped Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:56:34.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 systemd[1]: Starting Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T20:56:34.979 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 podman[29177]: 2024-04-25 20:56:34.885471398 +0000 UTC m=+0.098895288 container create 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T20:56:34.979 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:34 smithi037 podman[29177]: 2024-04-25 20:56:34.804838083 +0000 UTC m=+0.018261951 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 podman[29177]: 2024-04-25 20:56:35.104234552 +0000 UTC m=+0.317658401 container init 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 podman[29177]: 2024-04-25 20:56:35.107988089 +0000 UTC m=+0.321411937 container start 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: pidfile_write: ignore empty --pid-file 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: load: jerasure load: lrc 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Git sha 0 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: DB SUMMARY 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: DB Session ID: AH7M2V4EGB3G443PUVJO 2024-04-25T20:56:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105678 ; 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.env: 0x55d7a690ccc0 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.info_log: 0x55d7a7433be0 2024-04-25T20:56:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.statistics: (nil) 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.use_fsync: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:56:35.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.db_log_dir: 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.wal_dir: 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.write_buffer_manager: 0x55d7a73c70e0 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:56:35.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.unordered_write: 0 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.row_cache: None 2024-04-25T20:56:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.wal_filter: None 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.wal_compression: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:56:35.430 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:56:35.431 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_open_files: -1 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Compression algorithms supported: 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kZSTD supported: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:56:35.432 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.merge_operator: 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_filter: None 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:56:35.433 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d7a7433dc0) 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x55d7a73f91f0 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-04-25T20:56:35.434 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-04-25T20:56:35.435 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression: NoCompression 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:56:35.436 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.num_levels: 7 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:56:35.437 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:56:35.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:56:35.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:56:35.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.ttl: 2592000 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:56:35.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 44997ff7-d248-4ab6-9b2b-3b525dd03889 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078595137746, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078595138771, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100785, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98673, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90891, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078595, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "44997ff7-d248-4ab6-9b2b-3b525dd03889", "db_session_id": "AH7M2V4EGB3G443PUVJO", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:56:35.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078595138855, "job": 1, "event": "recovery_finished"} 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d7a750e000 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: DB pointer 0x55d7a74f8000 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:56:35.443 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 100.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 100.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:56:35.444 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 30.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 30.82 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x55d7a73f91f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:56:35.445 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???) e1 preinit fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).mds e1 new map 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).mds e1 print_map 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:56:35.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 bash[29177]: 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 systemd[1]: Started Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: monmap epoch 1 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: last_changed 2024-04-25T20:56:26.923204+0000 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: fsmap 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T20:56:35.447 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:35 smithi037 ceph-mon[29195]: mgrmap e1: no daemons active 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.672+0000 7f3bc0704700 1 Processor -- start 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.672+0000 7f3bc0704700 1 -- start start 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.672+0000 7f3bc0704700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.672+0000 7f3bc0704700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8108f00 con 0x7f3bb8108510 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.672+0000 7f3bbe4a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbe4a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57924/0 (socket says 172.21.15.37:57924) 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbe4a0700 1 -- 172.21.15.37:0/2591660717 learned_addr learned my addr 172.21.15.37:0/2591660717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbe4a0700 1 -- 172.21.15.37:0/2591660717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb81096c0 con 0x7f3bb8108510 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbe4a0700 1 --2- 172.21.15.37:0/2591660717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1671323fe96f9f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbd49e700 1 -- 172.21.15.37:0/2591660717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4004160 con 0x7f3bb8108510 2024-04-25T20:56:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbd49e700 1 -- 172.21.15.37:0/2591660717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3bb4004300 con 0x7f3bb8108510 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.673+0000 7f3bbd49e700 1 -- 172.21.15.37:0/2591660717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb4011af0 con 0x7f3bb8108510 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 -- 172.21.15.37:0/2591660717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 msgr2=0x7f3bb8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 --2- 172.21.15.37:0/2591660717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40098a0 tx=0x7f3bb4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 -- 172.21.15.37:0/2591660717 shutdown_connections 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 --2- 172.21.15.37:0/2591660717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb8108930 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 -- 172.21.15.37:0/2591660717 >> 172.21.15.37:0/2591660717 conn(0x7f3bb8103b70 msgr2=0x7f3bb8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 -- 172.21.15.37:0/2591660717 shutdown_connections 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.674+0000 7f3bc0704700 1 -- 172.21.15.37:0/2591660717 wait complete. 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bc0704700 1 Processor -- start 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bc0704700 1 -- start start 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bc0704700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bc0704700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb810ca70 con 0x7f3bb8108510 2024-04-25T20:56:36.336 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bbe4a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bbe4a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57928/0 (socket says 172.21.15.37:57928) 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bbe4a0700 1 -- 172.21.15.37:0/1192634896 learned_addr learned my addr 172.21.15.37:0/1192634896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bbe4a0700 1 -- 172.21.15.37:0/1192634896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb807e070 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bbe4a0700 1 --2- 172.21.15.37:0/1192634896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4003f40 tx=0x7f3bb4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb40038b0 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3bb4003a50 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.675+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb807c2b0 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.676+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bb401a5c0 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.676+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb807c790 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.676+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3bb401ac50 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.676+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3bb4019700 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.676+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb80640a0 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.679+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3bb4019900 con 0x7f3bb8108510 2024-04-25T20:56:36.337 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.751+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3bb8195de0 con 0x7f3bb8108510 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.753+0000 7f3baf7fe700 1 -- 172.21.15.37:0/1192634896 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3bb4019ae0 con 0x7f3bb8108510 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 msgr2=0x7f3bb807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 --2- 172.21.15.37:0/1192634896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4003f40 tx=0x7f3bb4011890 comp rx=0 tx=0).stop 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 shutdown_connections 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 --2- 172.21.15.37:0/1192634896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb8108510 0x7f3bb807db30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 >> 172.21.15.37:0/1192634896 conn(0x7f3bb8103b70 msgr2=0x7f3bb81048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 shutdown_connections 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:35.754+0000 7f3bc0704700 1 -- 172.21.15.37:0/1192634896 wait complete. 2024-04-25T20:56:36.338 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T20:56:36.352 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:56:36.352 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-04-25T20:56:36.352 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T20:56:36.353 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T20:56:36.606 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x 2024-04-25T20:56:36.606 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T20:56:36.758 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968.target.wants/ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1788dafa-0346-11ef-bc93-c7b262605968@.service. 2024-04-25T20:56:37.094 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:36 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1192634896' entity='client.admin' 2024-04-25T20:56:37.280 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:56:37.280 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T20:56:37.288 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:56:37.288 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:56:37.288 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T20:56:37.289 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:56:37.300 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:56:37.300 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-04-25T20:56:37.306 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-04-25T20:56:37.361 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:37 smithi037 systemd[1]: Started Ceph mgr.x for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:56:37.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:37.360+0000 7f46ce124200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:56:37.645 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:37.556+0000 7f46ce124200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:56:37.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:37 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1657211770' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:37.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:37.662+0000 7f46ce124200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:56:38.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:37.994+0000 7f46ce124200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:56:38.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:38.083+0000 7f46ce124200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:56:38.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:56:38.464 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:56:38.465 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:56:38.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:56:38.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:56:30.056166+0000", 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.746+0000 7fbd3779f700 1 Processor -- start 2024-04-25T20:56:38.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3779f700 1 -- start start 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3779f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3779f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd30107160 con 0x7fbd30106770 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3553b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3553b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57948/0 (socket says 172.21.15.37:57948) 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3553b700 1 -- 172.21.15.37:0/4039024918 learned_addr learned my addr 172.21.15.37:0/4039024918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.747+0000 7fbd3553b700 1 -- 172.21.15.37:0/4039024918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd301072e0 con 0x7fbd30106770 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd3553b700 1 --2- 172.21.15.37:0/4039024918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098a0 tx=0x7fbd2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82580ba0bb68cf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd27fff700 1 -- 172.21.15.37:0/4039024918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c004160 con 0x7fbd30106770 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd27fff700 1 -- 172.21.15.37:0/4039024918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd2c004300 con 0x7fbd30106770 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd27fff700 1 -- 172.21.15.37:0/4039024918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c011af0 con 0x7fbd30106770 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd3779f700 1 -- 172.21.15.37:0/4039024918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 msgr2=0x7fbd30106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.748+0000 7fbd3779f700 1 --2- 172.21.15.37:0/4039024918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c0098a0 tx=0x7fbd2c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 -- 172.21.15.37:0/4039024918 shutdown_connections 2024-04-25T20:56:38.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 --2- 172.21.15.37:0/4039024918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30106b90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 -- 172.21.15.37:0/4039024918 >> 172.21.15.37:0/4039024918 conn(0x7fbd30102110 msgr2=0x7fbd30104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 -- 172.21.15.37:0/4039024918 shutdown_connections 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 -- 172.21.15.37:0/4039024918 wait complete. 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 Processor -- start 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.749+0000 7fbd3779f700 1 -- start start 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3779f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3779f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3010a650 con 0x7fbd30106770 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3553b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3553b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57950/0 (socket says 172.21.15.37:57950) 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3553b700 1 -- 172.21.15.37:0/1657211770 learned_addr learned my addr 172.21.15.37:0/1657211770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3553b700 1 -- 172.21.15.37:0/1657211770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd301095b0 con 0x7fbd30106770 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3553b700 1 --2- 172.21.15.37:0/1657211770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009520 tx=0x7fbd2c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c0039c0 con 0x7fbd30106770 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd2c003b60 con 0x7fbd30106770 2024-04-25T20:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd301077a0 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd2c01a580 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.750+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd30107ce0 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.751+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd2c01abc0 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.751+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd2c019700 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.751+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd300640a0 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.755+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbd2c003d10 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.829+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd30108430 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.829+0000 7fbd267fc700 1 -- 172.21.15.37:0/1657211770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbd2c004300 con 0x7fbd30106770 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 msgr2=0x7fbd30109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 --2- 172.21.15.37:0/1657211770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009520 tx=0x7fbd2c011b70 comp rx=0 tx=0).stop 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 shutdown_connections 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 --2- 172.21.15.37:0/1657211770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbd30106770 0x7fbd30109070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 >> 172.21.15.37:0/1657211770 conn(0x7fbd30102110 msgr2=0x7fbd30102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 shutdown_connections 2024-04-25T20:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:37.831+0000 7fbd3779f700 1 -- 172.21.15.37:0/1657211770 wait complete. 2024-04-25T20:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-04-25T20:56:39.715 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:39.348+0000 7f46ce124200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:56:39.715 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:39.523+0000 7f46ce124200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:56:39.715 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:39.714+0000 7f46ce124200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:56:40.320 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:40.012+0000 7f46ce124200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:56:40.320 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:40.107+0000 7f46ce124200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:56:40.320 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:40.318+0000 7f46ce124200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:56:40.593 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:40.522+0000 7f46ce124200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:56:41.160 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:41 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/800338320' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:41.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:41.158+0000 7f46ce124200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:56:41.640 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:56:41.641 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:56:41.642 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:56:41.643 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:56:41.644 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:56:30.056166+0000", 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:56:41.645 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.911+0000 7f082c970700 1 Processor -- start 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.911+0000 7f082c970700 1 -- start start 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082c970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082c970700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f082807c830 con 0x7f082807de40 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41596/0 (socket says 172.21.15.37:41596) 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082659c700 1 -- 172.21.15.37:0/1278041231 learned_addr learned my addr 172.21.15.37:0/1278041231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082659c700 1 -- 172.21.15.37:0/1278041231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082807c9b0 con 0x7f082807de40 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.912+0000 7f082659c700 1 --2- 172.21.15.37:0/1278041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08180098a0 tx=0x7f0818009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcaf5aa2c63c2a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082559a700 1 -- 172.21.15.37:0/1278041231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0818004160 con 0x7f082807de40 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082559a700 1 -- 172.21.15.37:0/1278041231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0818004300 con 0x7f082807de40 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082559a700 1 -- 172.21.15.37:0/1278041231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0818011af0 con 0x7f082807de40 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 -- 172.21.15.37:0/1278041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 msgr2=0x7f082807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 --2- 172.21.15.37:0/1278041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f08180098a0 tx=0x7f0818009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.646 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 -- 172.21.15.37:0/1278041231 shutdown_connections 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 --2- 172.21.15.37:0/1278041231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082807c2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 -- 172.21.15.37:0/1278041231 >> 172.21.15.37:0/1278041231 conn(0x7f0828103ae0 msgr2=0x7f0828105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 -- 172.21.15.37:0/1278041231 shutdown_connections 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.913+0000 7f082c970700 1 -- 172.21.15.37:0/1278041231 wait complete. 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.914+0000 7f082c970700 1 Processor -- start 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.914+0000 7f082c970700 1 -- start start 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.914+0000 7f082c970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.914+0000 7f082c970700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f082810ea50 con 0x7f082807de40 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.914+0000 7f082659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41600/0 (socket says 172.21.15.37:41600) 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082659c700 1 -- 172.21.15.37:0/800338320 learned_addr learned my addr 172.21.15.37:0/800338320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082659c700 1 -- 172.21.15.37:0/800338320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082819eec0 con 0x7f082807de40 2024-04-25T20:56:41.647 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082659c700 1 --2- 172.21.15.37:0/800338320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0818009fb0 tx=0x7f0818011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08180036a0 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f081801a430 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0818019550 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082819f150 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281a1e40 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0818019be0 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.915+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0818024730 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.916+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08280640a0 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.918+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0818043e50 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.992+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f082802df80 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.993+0000 7f081f7fe700 1 -- 172.21.15.37:0/800338320 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0818032070 con 0x7f082807de40 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 msgr2=0x7f082819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 --2- 172.21.15.37:0/800338320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0818009fb0 tx=0x7f0818011940 comp rx=0 tx=0).stop 2024-04-25T20:56:41.648 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 shutdown_connections 2024-04-25T20:56:41.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 --2- 172.21.15.37:0/800338320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f082807de40 0x7f082819e980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 >> 172.21.15.37:0/800338320 conn(0x7f0828103ae0 msgr2=0x7f0828104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:41.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 shutdown_connections 2024-04-25T20:56:41.649 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:40.994+0000 7f082c970700 1 -- 172.21.15.37:0/800338320 wait complete. 2024-04-25T20:56:41.649 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-04-25T20:56:41.911 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:41.597+0000 7f46ce124200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:56:41.911 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:41.698+0000 7f46ce124200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:56:41.911 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:41.910+0000 7f46ce124200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:56:42.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:42.453+0000 7f46ce124200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:56:42.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:42.548+0000 7f46ce124200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:56:43.162 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:42.801+0000 7f46ce124200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:56:43.162 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:42.941+0000 7f46ce124200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:56:43.163 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:43.039+0000 7f46ce124200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:56:43.163 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:43.161+0000 7f46ce124200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:56:43.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:43.474+0000 7f46ce124200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:56:44.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:44 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/852904049' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:44.386 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:44.156+0000 7f46ce124200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:56:44.386 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:44.384+0000 7f46ce124200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:56:44.726 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:44.726 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:56:44.726 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T20:56:44.726 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:56:44.727 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:56:44.728 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:56:30.056166+0000", 2024-04-25T20:56:44.730 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a7e152700 1 Processor -- start 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a7e152700 1 -- start start 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a7e152700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a7e152700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7807be40 con 0x7f6a78106e10 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41604/0 (socket says 172.21.15.37:41604) 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.028+0000 7f6a777fe700 1 -- 172.21.15.37:0/1509366266 learned_addr learned my addr 172.21.15.37:0/1509366266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.029+0000 7f6a777fe700 1 -- 172.21.15.37:0/1509366266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a78107770 con 0x7f6a78106e10 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.029+0000 7f6a777fe700 1 --2- 172.21.15.37:0/1509366266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098a0 tx=0x7f6a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6827d89aff5da0e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:44.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.029+0000 7f6a767fc700 1 -- 172.21.15.37:0/1509366266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a68004160 con 0x7f6a78106e10 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.029+0000 7f6a767fc700 1 -- 172.21.15.37:0/1509366266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a68004300 con 0x7f6a78106e10 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.029+0000 7f6a767fc700 1 -- 172.21.15.37:0/1509366266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a68011af0 con 0x7f6a78106e10 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 -- 172.21.15.37:0/1509366266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 msgr2=0x7f6a78107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 --2- 172.21.15.37:0/1509366266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098a0 tx=0x7f6a68009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 -- 172.21.15.37:0/1509366266 shutdown_connections 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 --2- 172.21.15.37:0/1509366266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a78107230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 -- 172.21.15.37:0/1509366266 >> 172.21.15.37:0/1509366266 conn(0x7f6a78102a10 msgr2=0x7f6a78104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 -- 172.21.15.37:0/1509366266 shutdown_connections 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.030+0000 7f6a7e152700 1 -- 172.21.15.37:0/1509366266 wait complete. 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a7e152700 1 Processor -- start 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a7e152700 1 -- start start 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a7e152700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a7e152700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a78108730 con 0x7f6a78106e10 2024-04-25T20:56:44.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41612/0 (socket says 172.21.15.37:41612) 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a777fe700 1 -- 172.21.15.37:0/852904049 learned_addr learned my addr 172.21.15.37:0/852904049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.031+0000 7f6a777fe700 1 -- 172.21.15.37:0/852904049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7819eb80 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a777fe700 1 --2- 172.21.15.37:0/852904049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a68011ac0 tx=0x7f6a68011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a68003970 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a68003b10 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a68024610 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7819ee10 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a781a1b00 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a68024bd0 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.032+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a68019700 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.033+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a780640a0 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.035+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6a6801a9e0 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.159+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6a7802e180 con 0x7f6a78106e10 2024-04-25T20:56:44.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.159+0000 7f6a74ff9700 1 -- 172.21.15.37:0/852904049 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6a6801a430 con 0x7f6a78106e10 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 msgr2=0x7f6a7819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 --2- 172.21.15.37:0/852904049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a68011ac0 tx=0x7f6a68011ba0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 shutdown_connections 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 --2- 172.21.15.37:0/852904049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a78106e10 0x7f6a7819e640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 >> 172.21.15.37:0/852904049 conn(0x7f6a78102a10 msgr2=0x7f6a78195220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 shutdown_connections 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:44.160+0000 7f6a7e152700 1 -- 172.21.15.37:0/852904049 wait complete. 2024-04-25T20:56:44.734 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-04-25T20:56:45.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: mgrmap e2: x(active, starting, since 0.00318401s) 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' 2024-04-25T20:56:45.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' 2024-04-25T20:56:45.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:45 smithi037 ceph-mon[29195]: from='mgr.14100 172.21.15.37:0/1230195469' entity='mgr.x' 2024-04-25T20:56:46.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:46 smithi037 ceph-mon[29195]: mgrmap e3: x(active, since 1.00704s) 2024-04-25T20:56:47.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:47 smithi037 ceph-mon[29195]: mgrmap e4: x(active, since 2s) 2024-04-25T20:56:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:50 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1113029715' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:56:55.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:56:55.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:56:55.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:56:55.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:56:55.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:56:55.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:56:55.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:56:55.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:56:30.056166+0000", 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:56:55.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cfd979700 1 Processor -- start 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cfd979700 1 -- start start 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cfd979700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cfd979700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf8108f00 con 0x7f6cf8108510 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39878/0 (socket says 172.21.15.37:39878) 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.678+0000 7f6cf6ffd700 1 -- 172.21.15.37:0/732219284 learned_addr learned my addr 172.21.15.37:0/732219284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cf6ffd700 1 -- 172.21.15.37:0/732219284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf81096c0 con 0x7f6cf8108510 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cf6ffd700 1 --2- 172.21.15.37:0/732219284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80098a0 tx=0x7f6ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594505819fc67fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cf5ffb700 1 -- 172.21.15.37:0/732219284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce8004160 con 0x7f6cf8108510 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cf5ffb700 1 -- 172.21.15.37:0/732219284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ce8004300 con 0x7f6cf8108510 2024-04-25T20:56:55.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cf5ffb700 1 -- 172.21.15.37:0/732219284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce8011af0 con 0x7f6cf8108510 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cfd979700 1 -- 172.21.15.37:0/732219284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 msgr2=0x7f6cf8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cfd979700 1 --2- 172.21.15.37:0/732219284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80098a0 tx=0x7f6ce8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.679+0000 7f6cfd979700 1 -- 172.21.15.37:0/732219284 shutdown_connections 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 --2- 172.21.15.37:0/732219284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf8108930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 -- 172.21.15.37:0/732219284 >> 172.21.15.37:0/732219284 conn(0x7f6cf8103b70 msgr2=0x7f6cf8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 -- 172.21.15.37:0/732219284 shutdown_connections 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 -- 172.21.15.37:0/732219284 wait complete. 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 Processor -- start 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 -- start start 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.680+0000 7f6cfd979700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfd979700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810ca70 con 0x7f6cf8108510 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cf6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39882/0 (socket says 172.21.15.37:39882) 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cf6ffd700 1 -- 172.21.15.37:0/1113029715 learned_addr learned my addr 172.21.15.37:0/1113029715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:55.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cf6ffd700 1 -- 172.21.15.37:0/1113029715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf819ccc0 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cf6ffd700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8003f40 tx=0x7f6ce8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce80038b0 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ce8003a50 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ce801a5c0 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf819cf50 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.681+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf819fbf0 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.682+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f6ce801a870 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.682+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf80640a0 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.683+0000 7f6cfc977700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 0x7f6ce003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.683+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ce804db90 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.683+0000 7f6cf67fc700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 0x7f6ce003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.683+0000 7f6cf67fc700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 0x7f6ce003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006fd0 tx=0x7f6cec006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.688+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ce8019c70 con 0x7f6cf8108510 2024-04-25T20:56:55.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.982+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6cf802dfb0 con 0x7f6cf8108510 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.983+0000 7f6cfc977700 1 -- 172.21.15.37:0/1113029715 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6ce802c9c0 con 0x7f6cf8108510 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 msgr2=0x7f6ce003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 0x7f6ce003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6cec006fd0 tx=0x7f6cec006e40 comp rx=0 tx=0).stop 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 msgr2=0x7f6cf819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8003f40 tx=0x7f6ce8011890 comp rx=0 tx=0).stop 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 shutdown_connections 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6ce0039560 0x7f6ce003ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 --2- 172.21.15.37:0/1113029715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6cf8108510 0x7f6cf819c780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 >> 172.21.15.37:0/1113029715 conn(0x7f6cf8103b70 msgr2=0x7f6cf81048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 shutdown_connections 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:49.985+0000 7f6cfd979700 1 -- 172.21.15.37:0/1113029715 wait complete. 2024-04-25T20:56:55.787 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-04-25T20:56:58.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:58 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3158130061' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:56:58.571 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d960e8700 1 Processor -- start 2024-04-25T20:56:58.572 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d960e8700 1 -- start start 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d960e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d960e8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90108f00 con 0x7f3d90108510 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39894/0 (socket says 172.21.15.37:39894) 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d8f7fe700 1 -- 172.21.15.37:0/1429440955 learned_addr learned my addr 172.21.15.37:0/1429440955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.777+0000 7f3d8f7fe700 1 -- 172.21.15.37:0/1429440955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d901096c0 con 0x7f3d90108510 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d8f7fe700 1 --2- 172.21.15.37:0/1429440955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3d800098a0 tx=0x7f3d80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6946e881b77b2e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d8e7fc700 1 -- 172.21.15.37:0/1429440955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d80004160 con 0x7f3d90108510 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d8e7fc700 1 -- 172.21.15.37:0/1429440955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d80004300 con 0x7f3d90108510 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d8e7fc700 1 -- 172.21.15.37:0/1429440955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d80011af0 con 0x7f3d90108510 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d960e8700 1 -- 172.21.15.37:0/1429440955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 msgr2=0x7f3d90108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.778+0000 7f3d960e8700 1 --2- 172.21.15.37:0/1429440955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3d800098a0 tx=0x7f3d80009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 -- 172.21.15.37:0/1429440955 shutdown_connections 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 --2- 172.21.15.37:0/1429440955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d90108930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.573 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 -- 172.21.15.37:0/1429440955 >> 172.21.15.37:0/1429440955 conn(0x7f3d90103b70 msgr2=0x7f3d90105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 -- 172.21.15.37:0/1429440955 shutdown_connections 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 -- 172.21.15.37:0/1429440955 wait complete. 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 Processor -- start 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.779+0000 7f3d960e8700 1 -- start start 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d960e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d960e8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9010ca70 con 0x7f3d90108510 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39908/0 (socket says 172.21.15.37:39908) 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8f7fe700 1 -- 172.21.15.37:0/3158130061 learned_addr learned my addr 172.21.15.37:0/3158130061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8f7fe700 1 -- 172.21.15.37:0/3158130061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9019ee90 con 0x7f3d90108510 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8f7fe700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d80003f40 tx=0x7f3d80011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d800038b0 con 0x7f3d90108510 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d80003a50 con 0x7f3d90108510 2024-04-25T20:56:58.574 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9019f120 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.780+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8001a5c0 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.781+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d901a1e10 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.781+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f3d8001a870 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.782+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d900640a0 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.782+0000 7f3d8cff9700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 0x7f3d7803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.782+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d8004da40 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.782+0000 7f3d8effd700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 0x7f3d7803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.782+0000 7f3d8effd700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 0x7f3d7803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d84006fd0 tx=0x7f3d84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.787+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d80019c70 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.994+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3d901a3160 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.995+0000 7f3d8cff9700 1 -- 172.21.15.37:0/3158130061 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f3d8004d460 con 0x7f3d90108510 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 msgr2=0x7f3d7803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 0x7f3d7803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d84006fd0 tx=0x7f3d84006e40 comp rx=0 tx=0).stop 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 msgr2=0x7f3d9019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.575 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d80003f40 tx=0x7f3d80011890 comp rx=0 tx=0).stop 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 shutdown_connections 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f3d78039560 0x7f3d7803ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 --2- 172.21.15.37:0/3158130061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d90108510 0x7f3d9019e950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.997+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 >> 172.21.15.37:0/3158130061 conn(0x7f3d90103b70 msgr2=0x7f3d90104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.998+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 shutdown_connections 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:57.998+0000 7f3d960e8700 1 -- 172.21.15.37:0/3158130061 wait complete. 2024-04-25T20:56:58.576 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-04-25T20:56:59.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/235301100' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T20:56:59.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setuser ceph since I am not root 2024-04-25T20:56:59.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setgroup ceph since I am not root 2024-04-25T20:56:59.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:59.436+0000 7fe3e4f32200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:56:59.850 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:59.628+0000 7fe3e4f32200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:56:59.850 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:56:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:56:59.734+0000 7fe3e4f32200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:56:59.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.933+0000 7f6a7c5a8700 1 Processor -- start 2024-04-25T20:56:59.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.933+0000 7f6a7c5a8700 1 -- start start 2024-04-25T20:56:59.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.933+0000 7f6a7c5a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:59.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.933+0000 7f6a7c5a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7407c8e0 con 0x7f6a7407de60 2024-04-25T20:56:59.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a7a344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a7a344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51028/0 (socket says 172.21.15.37:51028) 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a7a344700 1 -- 172.21.15.37:0/2994452163 learned_addr learned my addr 172.21.15.37:0/2994452163 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a7a344700 1 -- 172.21.15.37:0/2994452163 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7407ca60 con 0x7f6a7407de60 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a7a344700 1 --2- 172.21.15.37:0/2994452163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098a0 tx=0x7f6a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292f7c5bb45d727e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a79342700 1 -- 172.21.15.37:0/2994452163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a70004160 con 0x7f6a7407de60 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a79342700 1 -- 172.21.15.37:0/2994452163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a70004300 con 0x7f6a7407de60 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.934+0000 7f6a79342700 1 -- 172.21.15.37:0/2994452163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a70011af0 con 0x7f6a7407de60 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/2994452163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 msgr2=0x7f6a7407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/2994452163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098a0 tx=0x7f6a70009bb0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/2994452163 shutdown_connections 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/2994452163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7407c310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/2994452163 >> 172.21.15.37:0/2994452163 conn(0x7f6a74103b70 msgr2=0x7f6a74105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/2994452163 shutdown_connections 2024-04-25T20:56:59.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.935+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/2994452163 wait complete. 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7c5a8700 1 Processor -- start 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7c5a8700 1 -- start start 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7c5a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7c5a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7410ea60 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7a344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7a344700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51038/0 (socket says 172.21.15.37:51038) 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7a344700 1 -- 172.21.15.37:0/235301100 learned_addr learned my addr 172.21.15.37:0/235301100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7a344700 1 -- 172.21.15.37:0/235301100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7419ee50 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.936+0000 7f6a7a344700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a70011ac0 tx=0x7f6a70011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.937+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a70003b20 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.937+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a7001a8c0 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.937+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7419f0e0 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.937+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a700248d0 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.937+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a741a1dd0 con 0x7f6a7407de60 2024-04-25T20:56:59.860 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f6a70022020 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a740640a0 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a6b7fe700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 0x7f6a6003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a7004d5f0 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a79b43700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 0x7f6a6003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.938+0000 7f6a79b43700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 0x7f6a6003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a64006fd0 tx=0x7f6a64006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:58.943+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a7002cdf0 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.212+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6a7402df80 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.263+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6a7001ade0 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.265+0000 7f6a6b7fe700 1 -- 172.21.15.37:0/235301100 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6a7002dc90 con 0x7f6a7407de60 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 msgr2=0x7f6a6003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 0x7f6a6003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6a64006fd0 tx=0x7f6a64006e40 comp rx=0 tx=0).stop 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 msgr2=0x7f6a7419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6a70011ac0 tx=0x7f6a70011ba0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.861 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 shutdown_connections 2024-04-25T20:56:59.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a600395b0 0x7f6a6003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 --2- 172.21.15.37:0/235301100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7407de60 0x7f6a7419e910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:59.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.266+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 >> 172.21.15.37:0/235301100 conn(0x7f6a74103b70 msgr2=0x7f6a74104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:59.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.267+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 shutdown_connections 2024-04-25T20:56:59.862 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:56:59.267+0000 7f6a7c5a8700 1 -- 172.21.15.37:0/235301100 wait complete. 2024-04-25T20:57:00.149 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:00.058+0000 7fe3e4f32200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:57:00.150 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:00.148+0000 7fe3e4f32200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:57:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:00 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/235301100' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T20:57:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:00 smithi037 ceph-mon[29195]: mgrmap e5: x(active, since 14s) 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:57:01.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3fd3d700 1 Processor -- start 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3fd3d700 1 -- start start 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3fd3d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3fd3d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3807bec0 con 0x7f6a38108f00 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3dad9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3dad9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51054/0 (socket says 172.21.15.37:51054) 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.288+0000 7f6a3dad9700 1 -- 172.21.15.37:0/130853115 learned_addr learned my addr 172.21.15.37:0/130853115 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.289+0000 7f6a3dad9700 1 -- 172.21.15.37:0/130853115 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3810b830 con 0x7f6a38108f00 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.289+0000 7f6a3dad9700 1 --2- 172.21.15.37:0/130853115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098a0 tx=0x7f6a34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2340e61ee684c7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.289+0000 7f6a3cad7700 1 -- 172.21.15.37:0/130853115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34004160 con 0x7f6a38108f00 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.289+0000 7f6a3cad7700 1 -- 172.21.15.37:0/130853115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a34004300 con 0x7f6a38108f00 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.289+0000 7f6a3cad7700 1 -- 172.21.15.37:0/130853115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34011af0 con 0x7f6a38108f00 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/130853115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 msgr2=0x7f6a3810b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/130853115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6a340098a0 tx=0x7f6a34009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/130853115 shutdown_connections 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/130853115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3810b2f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/130853115 >> 172.21.15.37:0/130853115 conn(0x7f6a381029b0 msgr2=0x7f6a38104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/130853115 shutdown_connections 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/130853115 wait complete. 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 Processor -- start 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.290+0000 7f6a3fd3d700 1 -- start start 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3fd3d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3fd3d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3810c800 con 0x7f6a38108f00 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3dad9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3dad9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51060/0 (socket says 172.21.15.37:51060) 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3dad9700 1 -- 172.21.15.37:0/3309968578 learned_addr learned my addr 172.21.15.37:0/3309968578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3dad9700 1 -- 172.21.15.37:0/3309968578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3819ebb0 con 0x7f6a38108f00 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3dad9700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009fb0 tx=0x7f6a34011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:01.200 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a340036a0 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6a3401a430 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3819ee40 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.291+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a34019550 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.292+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a1b30 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.292+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a380640a0 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.293+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f6a34019850 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.293+0000 7f6a2effd700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 0x7f6a2403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.293+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6a3404da70 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.293+0000 7f6a3d2d8700 1 -- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 msgr2=0x7f6a2403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.293+0000 7f6a3d2d8700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 0x7f6a2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.298+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a3404d490 con 0x7f6a38108f00 2024-04-25T20:57:01.201 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.494+0000 7f6a3d2d8700 1 -- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 msgr2=0x7f6a2403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.494+0000 7f6a3d2d8700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 0x7f6a2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.615+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f6a381a2170 con 0x7f6a38108f00 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.615+0000 7f6a2effd700 1 -- 172.21.15.37:0/3309968578 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f6a3400bde0 con 0x7f6a38108f00 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 msgr2=0x7f6a2403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 0x7f6a2403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 msgr2=0x7f6a3819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009fb0 tx=0x7f6a34011940 comp rx=0 tx=0).stop 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 shutdown_connections 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f6a24039560 0x7f6a2403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 --2- 172.21.15.37:0/3309968578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38108f00 0x7f6a3819e670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 >> 172.21.15.37:0/3309968578 conn(0x7f6a381029b0 msgr2=0x7f6a38103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 shutdown_connections 2024-04-25T20:57:01.202 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:00.618+0000 7f6a3fd3d700 1 -- 172.21.15.37:0/3309968578 wait complete. 2024-04-25T20:57:01.203 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-25T20:57:01.203 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-04-25T20:57:01.448 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:01 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3309968578' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:57:01.448 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:01.417+0000 7fe3e4f32200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:57:01.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:01.601+0000 7fe3e4f32200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:02.081 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:01.780+0000 7fe3e4f32200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:57:02.082 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:02.080+0000 7fe3e4f32200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:57:02.388 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:02.176+0000 7fe3e4f32200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:57:02.388 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:02.386+0000 7fe3e4f32200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:57:02.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:02.586+0000 7fe3e4f32200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:03.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:03.210+0000 7fe3e4f32200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:57:03.957 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:03.648+0000 7fe3e4f32200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:57:03.958 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:03.748+0000 7fe3e4f32200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:57:03.958 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:03.956+0000 7fe3e4f32200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:57:04.850 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:04.500+0000 7fe3e4f32200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:57:04.850 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:04.596+0000 7fe3e4f32200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:57:05.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:04.848+0000 7fe3e4f32200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:57:05.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:04.987+0000 7fe3e4f32200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:57:05.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:05.086+0000 7fe3e4f32200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:57:05.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:05.206+0000 7fe3e4f32200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:05.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:05.520+0000 7fe3e4f32200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:57:06.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:06.146+0000 7fe3e4f32200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:57:06.424 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:06.371+0000 7fe3e4f32200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:57:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T20:57:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:57:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T20:57:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: mgrmap e6: x(active, starting, since 0.00332516s) 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:06.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:06 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:57:07.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:07 smithi037 ceph-mon[29195]: Found migration_current of "None". Setting to last migration. 2024-04-25T20:57:07.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:07 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:57:07.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:07 smithi037 ceph-mon[29195]: mgrmap e7: x(active, since 1.00533s) 2024-04-25T20:57:07.939 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:57:07.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T20:57:07.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a4a010700 1 Processor -- start 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a4a010700 1 -- start start 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a4a010700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a4a010700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4407be60 con 0x7f2a44108f60 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51072/0 (socket says 172.21.15.37:51072) 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.655+0000 7f2a437fe700 1 -- 172.21.15.37:0/700731103 learned_addr learned my addr 172.21.15.37:0/700731103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.656+0000 7f2a437fe700 1 -- 172.21.15.37:0/700731103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4410b890 con 0x7f2a44108f60 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.656+0000 7f2a437fe700 1 --2- 172.21.15.37:0/700731103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098a0 tx=0x7f2a2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42da446f041de042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.656+0000 7f2a427fc700 1 -- 172.21.15.37:0/700731103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a2c004160 con 0x7f2a44108f60 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.656+0000 7f2a427fc700 1 -- 172.21.15.37:0/700731103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a2c004300 con 0x7f2a44108f60 2024-04-25T20:57:07.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.656+0000 7f2a427fc700 1 -- 172.21.15.37:0/700731103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a2c011af0 con 0x7f2a44108f60 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 -- 172.21.15.37:0/700731103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 msgr2=0x7f2a4410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 --2- 172.21.15.37:0/700731103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098a0 tx=0x7f2a2c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 -- 172.21.15.37:0/700731103 shutdown_connections 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 --2- 172.21.15.37:0/700731103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4410b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 -- 172.21.15.37:0/700731103 >> 172.21.15.37:0/700731103 conn(0x7f2a441029b0 msgr2=0x7f2a44104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 -- 172.21.15.37:0/700731103 shutdown_connections 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.657+0000 7f2a4a010700 1 -- 172.21.15.37:0/700731103 wait complete. 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a4a010700 1 Processor -- start 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a4a010700 1 -- start start 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a4a010700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a4a010700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4410c800 con 0x7f2a44108f60 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:07.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a437fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51080/0 (socket says 172.21.15.37:51080) 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a437fe700 1 -- 172.21.15.37:0/2359635046 learned_addr learned my addr 172.21.15.37:0/2359635046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.658+0000 7f2a437fe700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4419c990 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a437fe700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009fb0 tx=0x7f2a2c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a2c0036a0 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4419cc20 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4419f910 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a2c01a430 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.659+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a2c019550 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f2a2c019800 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a40ff9700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2a3003c180 con 0x7f2a300395b0 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a2c04d8e0 con 0x7f2a44108f60 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a42ffd700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:07.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.660+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.861+0000 7f2a42ffd700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:01.861+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:02.262+0000 7f2a42ffd700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:02.262+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:03.063+0000 7f2a42ffd700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:03.063+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:04.664+0000 7f2a42ffd700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1093601384 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:04.664+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:06.374+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f2a2c019b40 con 0x7f2a44108f60 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:06.374+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 msgr2=0x7f2a3003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:06.374+0000 7f2a40ff9700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.375+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f2a2c04f3f0 con 0x7f2a44108f60 2024-04-25T20:57:07.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.375+0000 7f2a40ff9700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 0x7f2a3003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.375+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2a2c012ea0 con 0x7f2a3003cea0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.376+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 0x7f2a3003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.376+0000 7f2a42ffd700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 0x7f2a3003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2a34003a10 tx=0x7f2a340092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.377+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2a2c012ea0 con 0x7f2a3003cea0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2a44069b90 con 0x7f2a3003cea0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a40ff9700 1 -- 172.21.15.37:0/2359635046 <== mgr.14120 v2:172.21.15.37:6800/3898721946 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2a44069b90 con 0x7f2a3003cea0 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 msgr2=0x7f2a3003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a4a010700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 0x7f2a3003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2a34003a10 tx=0x7f2a340092b0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 msgr2=0x7f2a4419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.384+0000 7f2a4a010700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009fb0 tx=0x7f2a2c011940 comp rx=0 tx=0).stop 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 shutdown_connections 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2a3003cea0 0x7f2a3003f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a44108f60 0x7f2a4419c450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 --2- 172.21.15.37:0/2359635046 >> [v2:172.21.15.37:6800/1093601384,v1:172.21.15.37:6801/1093601384] conn(0x7f2a300395b0 0x7f2a3003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 >> 172.21.15.37:0/2359635046 conn(0x7f2a441029b0 msgr2=0x7f2a441035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:07.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 shutdown_connections 2024-04-25T20:57:07.952 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:07.385+0000 7f2a4a010700 1 -- 172.21.15.37:0/2359635046 wait complete. 2024-04-25T20:57:07.952 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-04-25T20:57:07.952 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-04-25T20:57:08.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:08 smithi037 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:57:08.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:08 smithi037 ceph-mon[29195]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:57:08.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:08 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:08.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:08 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.366+0000 7f7b7fed6700 1 Processor -- start 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.366+0000 7f7b7fed6700 1 -- start start 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.366+0000 7f7b7fed6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.366+0000 7f7b7fed6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b78109370 con 0x7f7b78108980 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7dc72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7dc72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51170/0 (socket says 172.21.15.37:51170) 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7dc72700 1 -- 172.21.15.37:0/3118772515 learned_addr learned my addr 172.21.15.37:0/3118772515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7dc72700 1 -- 172.21.15.37:0/3118772515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b781075e0 con 0x7f7b78108980 2024-04-25T20:57:09.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7dc72700 1 --2- 172.21.15.37:0/3118772515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400bb40 tx=0x7f7b7400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6a8fdb726ca174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7cc70700 1 -- 172.21.15.37:0/3118772515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b7400dbe0 con 0x7f7b78108980 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7cc70700 1 -- 172.21.15.37:0/3118772515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b740044c0 con 0x7f7b78108980 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.367+0000 7f7b7cc70700 1 -- 172.21.15.37:0/3118772515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b74010470 con 0x7f7b78108980 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3118772515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 msgr2=0x7f7b78108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3118772515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400bb40 tx=0x7f7b7400be50 comp rx=0 tx=0).stop 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3118772515 shutdown_connections 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3118772515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b78108da0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3118772515 >> 172.21.15.37:0/3118772515 conn(0x7f7b78102110 msgr2=0x7f7b78104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3118772515 shutdown_connections 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3118772515 wait complete. 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.368+0000 7f7b7fed6700 1 Processor -- start 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7fed6700 1 -- start start 2024-04-25T20:57:09.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7fed6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7fed6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b781064d0 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7dc72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7dc72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51184/0 (socket says 172.21.15.37:51184) 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7dc72700 1 -- 172.21.15.37:0/3828476159 learned_addr learned my addr 172.21.15.37:0/3828476159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7dc72700 1 -- 172.21.15.37:0/3828476159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7819a750 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7dc72700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d650 tx=0x7f7b74003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b74027070 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.369+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7819a9e0 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.370+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b74010c00 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.370+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b74022ba0 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.370+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7819d6d0 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.370+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7b7402a070 con 0x7f7b78108980 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.371+0000 7f7b6affd700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 0x7f7b6403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.371+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b7401f070 con 0x7f7b78108980 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.371+0000 7f7b7d471700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 0x7f7b6403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.371+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b780640a0 con 0x7f7b78108980 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.371+0000 7f7b7d471700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 0x7f7b6403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006fd0 tx=0x7f7b6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.376+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b74031030 con 0x7f7b78108980 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.569+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7b74022410 con 0x7f7b78108980 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.605+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7b78062e80 con 0x7f7b640394a0 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.610+0000 7f7b6affd700 1 -- 172.21.15.37:0/3828476159 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7b78062e80 con 0x7f7b640394a0 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 msgr2=0x7f7b6403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 0x7f7b6403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006fd0 tx=0x7f7b6c006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 msgr2=0x7f7b7819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400d650 tx=0x7f7b74003ff0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 shutdown_connections 2024-04-25T20:57:09.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f7b640394a0 0x7f7b6403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 --2- 172.21.15.37:0/3828476159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b78108980 0x7f7b7819a210 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 >> 172.21.15.37:0/3828476159 conn(0x7f7b78102110 msgr2=0x7f7b78190830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:09.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 shutdown_connections 2024-04-25T20:57:09.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:08.613+0000 7f7b7fed6700 1 -- 172.21.15.37:0/3828476159 wait complete. 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: mgrmap e8: x(active, since 2s) 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:09] ENGINE Bus STARTING 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:09] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:09] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T20:57:09.817 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:09] ENGINE Bus STARTED 2024-04-25T20:57:09.818 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:09 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.597+0000 7fb98eda0700 1 Processor -- start 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.597+0000 7fb98eda0700 1 -- start start 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.597+0000 7fb98eda0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.597+0000 7fb98eda0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb98807c830 con 0x7fb98807de40 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb98cb3c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb98cb3c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38122/0 (socket says 172.21.15.37:38122) 2024-04-25T20:57:10.442 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb98cb3c700 1 -- 172.21.15.37:0/2925287054 learned_addr learned my addr 172.21.15.37:0/2925287054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb98cb3c700 1 -- 172.21.15.37:0/2925287054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98807c9b0 con 0x7fb98807de40 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb98cb3c700 1 --2- 172.21.15.37:0/2925287054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb980009e80 tx=0x7fb98000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab26f65c69624fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb9877fe700 1 -- 172.21.15.37:0/2925287054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980017070 con 0x7fb98807de40 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb9877fe700 1 -- 172.21.15.37:0/2925287054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb98000b8c0 con 0x7fb98807de40 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.598+0000 7fb9877fe700 1 -- 172.21.15.37:0/2925287054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980004490 con 0x7fb98807de40 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 -- 172.21.15.37:0/2925287054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 msgr2=0x7fb98807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 --2- 172.21.15.37:0/2925287054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb980009e80 tx=0x7fb98000b330 comp rx=0 tx=0).stop 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 -- 172.21.15.37:0/2925287054 shutdown_connections 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 --2- 172.21.15.37:0/2925287054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98807c2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 -- 172.21.15.37:0/2925287054 >> 172.21.15.37:0/2925287054 conn(0x7fb988103ab0 msgr2=0x7fb988105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 -- 172.21.15.37:0/2925287054 shutdown_connections 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 -- 172.21.15.37:0/2925287054 wait complete. 2024-04-25T20:57:10.443 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.599+0000 7fb98eda0700 1 Processor -- start 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98eda0700 1 -- start start 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98eda0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98eda0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb98810ea60 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98cb3c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98cb3c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38124/0 (socket says 172.21.15.37:38124) 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98cb3c700 1 -- 172.21.15.37:0/1468478522 learned_addr learned my addr 172.21.15.37:0/1468478522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98cb3c700 1 -- 172.21.15.37:0/1468478522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98819ee60 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98cb3c700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98000b7b0 tx=0x7fb98000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980020020 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb980012430 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98819f0f0 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.600+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980017040 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.601+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9881a1de0 con 0x7fb98807de40 2024-04-25T20:57:10.444 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb980023020 con 0x7fb98807de40 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb988069b90 con 0x7fb98807de40 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb985ffb700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 0x7fb97003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb98000f050 con 0x7fb98807de40 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb987fff700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 0x7fb97003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.602+0000 7fb987fff700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 0x7fb97003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb97c006fd0 tx=0x7fb97c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.607+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb98002a020 con 0x7fb98807de40 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.816+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb98802dff0 con 0x7fb970039600 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.816+0000 7fb985ffb700 1 -- 172.21.15.37:0/1468478522 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb98802dff0 con 0x7fb970039600 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 msgr2=0x7fb97003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 0x7fb97003bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb97c006fd0 tx=0x7fb97c006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 msgr2=0x7fb98819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98000b7b0 tx=0x7fb98000bda0 comp rx=0 tx=0).stop 2024-04-25T20:57:10.445 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 shutdown_connections 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fb970039600 0x7fb97003bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 --2- 172.21.15.37:0/1468478522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb98807de40 0x7fb98819e920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.818+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 >> 172.21.15.37:0/1468478522 conn(0x7fb988103ab0 msgr2=0x7fb988104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.819+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 shutdown_connections 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:09.819+0000 7fb98eda0700 1 -- 172.21.15.37:0/1468478522 wait complete. 2024-04-25T20:57:10.446 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-04-25T20:57:10.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:10 smithi037 ceph-mon[29195]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:11.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-mon[29195]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:11.850 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-mon[29195]: Generating ssh key... 2024-04-25T20:57:12.138 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: Generating public/private rsa key pair. 2024-04-25T20:57:12.138 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: Your identification has been saved in /tmp/tmpljae1uti/key. 2024-04-25T20:57:12.138 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: Your public key has been saved in /tmp/tmpljae1uti/key.pub. 2024-04-25T20:57:12.138 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: The key fingerprint is: 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: SHA256:UWkLF8j5QONtRsrwJY19qwxxemE9wy4UVQvh+WxHB34 ceph-1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: The key's randomart image is: 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: +---[RSA 3072]----+ 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | .o+**B+oo | 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | =OO@.*+ o | 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | *%+=o+o E| 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | o+= oo o.| 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | S+ o + .| 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | o . . | 2024-04-25T20:57:12.139 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | | 2024-04-25T20:57:12.140 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | | 2024-04-25T20:57:12.140 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: | | 2024-04-25T20:57:12.140 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: +----[SHA256]-----+ 2024-04-25T20:57:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.897+0000 7f3b9f1a5700 1 Processor -- start 2024-04-25T20:57:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.897+0000 7f3b9f1a5700 1 -- start start 2024-04-25T20:57:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9f1a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9f1a5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9807be80 con 0x7f3b98108f60 2024-04-25T20:57:12.379 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9cf41700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9cf41700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38132/0 (socket says 172.21.15.37:38132) 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9cf41700 1 -- 172.21.15.37:0/39886802 learned_addr learned my addr 172.21.15.37:0/39886802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9cf41700 1 -- 172.21.15.37:0/39886802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9810b890 con 0x7f3b98108f60 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b9cf41700 1 --2- 172.21.15.37:0/39886802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098a0 tx=0x7f3b8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c414c14a4cec1702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.898+0000 7f3b977fe700 1 -- 172.21.15.37:0/39886802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b8c004160 con 0x7f3b98108f60 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b977fe700 1 -- 172.21.15.37:0/39886802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b8c004300 con 0x7f3b98108f60 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b977fe700 1 -- 172.21.15.37:0/39886802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b8c011af0 con 0x7f3b98108f60 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/39886802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 msgr2=0x7f3b9810b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/39886802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0098a0 tx=0x7f3b8c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/39886802 shutdown_connections 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/39886802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b9810b350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/39886802 >> 172.21.15.37:0/39886802 conn(0x7f3b98102970 msgr2=0x7f3b98104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:12.380 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/39886802 shutdown_connections 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.899+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/39886802 wait complete. 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9f1a5700 1 Processor -- start 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9f1a5700 1 -- start start 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9f1a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9f1a5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810c850 con 0x7f3b98108f60 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9cf41700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9cf41700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38148/0 (socket says 172.21.15.37:38148) 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9cf41700 1 -- 172.21.15.37:0/2359038471 learned_addr learned my addr 172.21.15.37:0/2359038471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.900+0000 7f3b9cf41700 1 -- 172.21.15.37:0/2359038471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b98108d20 con 0x7f3b98108f60 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b9cf41700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c011ac0 tx=0x7f3b8c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b8c003970 con 0x7f3b98108f60 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b8c003b10 con 0x7f3b98108f60 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b98106f10 con 0x7f3b98108f60 2024-04-25T20:57:12.381 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b8c024610 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.901+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b98107450 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.902+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3b8c024890 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.902+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b980640a0 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.902+0000 7f3b95ffb700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 0x7f3b8803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.902+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3b8c04da50 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.902+0000 7f3b97fff700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 0x7f3b8803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.903+0000 7f3b97fff700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 0x7f3b8803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006fd0 tx=0x7f3b84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:10.908+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b8c02d910 con 0x7f3b98108f60 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.122+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3b98103c30 con 0x7f3b880395b0 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.855+0000 7f3b95ffb700 1 -- 172.21.15.37:0/2359038471 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b98103c30 con 0x7f3b880395b0 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 msgr2=0x7f3b8803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 0x7f3b8803ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b84006fd0 tx=0x7f3b84006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 msgr2=0x7f3b981087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.382 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c011ac0 tx=0x7f3b8c011ba0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 shutdown_connections 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3b880395b0 0x7f3b8803ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 --2- 172.21.15.37:0/2359038471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b98108f60 0x7f3b981087e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 >> 172.21.15.37:0/2359038471 conn(0x7f3b98102970 msgr2=0x7f3b98103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 shutdown_connections 2024-04-25T20:57:12.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:11.857+0000 7f3b9f1a5700 1 -- 172.21.15.37:0/2359038471 wait complete. 2024-04-25T20:57:13.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:12 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:13.033 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:12 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.798+0000 7fa90953c700 1 Processor -- start 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.798+0000 7fa90953c700 1 -- start start 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa90953c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa90953c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa904108f00 con 0x7fa904108510 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa902ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:13.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa902ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38152/0 (socket says 172.21.15.37:38152) 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa902ffd700 1 -- 172.21.15.37:0/3364716655 learned_addr learned my addr 172.21.15.37:0/3364716655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa902ffd700 1 -- 172.21.15.37:0/3364716655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9041096c0 con 0x7fa904108510 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa902ffd700 1 --2- 172.21.15.37:0/3364716655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0098a0 tx=0x7fa8ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fb6de76feb9191 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa901ffb700 1 -- 172.21.15.37:0/3364716655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8ec004160 con 0x7fa904108510 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa901ffb700 1 -- 172.21.15.37:0/3364716655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa8ec004300 con 0x7fa904108510 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.799+0000 7fa901ffb700 1 -- 172.21.15.37:0/3364716655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8ec011af0 con 0x7fa904108510 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 -- 172.21.15.37:0/3364716655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 msgr2=0x7fa904108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 --2- 172.21.15.37:0/3364716655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0098a0 tx=0x7fa8ec009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 -- 172.21.15.37:0/3364716655 shutdown_connections 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 --2- 172.21.15.37:0/3364716655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa904108930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 -- 172.21.15.37:0/3364716655 >> 172.21.15.37:0/3364716655 conn(0x7fa904103b70 msgr2=0x7fa904105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 -- 172.21.15.37:0/3364716655 shutdown_connections 2024-04-25T20:57:13.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.800+0000 7fa90953c700 1 -- 172.21.15.37:0/3364716655 wait complete. 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa90953c700 1 Processor -- start 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa90953c700 1 -- start start 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa90953c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa90953c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa90410ca70 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa902ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa902ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38156/0 (socket says 172.21.15.37:38156) 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.801+0000 7fa902ffd700 1 -- 172.21.15.37:0/1355905080 learned_addr learned my addr 172.21.15.37:0/1355905080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa902ffd700 1 -- 172.21.15.37:0/1355905080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90407e070 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa902ffd700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec003f40 tx=0x7fa8ec011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8ec0038b0 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa90407c2b0 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa8ec003a50 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8ec01a5c0 con 0x7fa904108510 2024-04-25T20:57:13.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.802+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90407c790 con 0x7fa904108510 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.803+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa8ec01a8c0 con 0x7fa904108510 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.803+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9040640a0 con 0x7fa904108510 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.803+0000 7fa8fbfff700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 0x7fa8f003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.803+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa8ec04db20 con 0x7fa904108510 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.803+0000 7fa9027fc700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 0x7fa8f003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.804+0000 7fa9027fc700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 0x7fa8f003ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006fd0 tx=0x7fa8f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:12.808+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8ec019c70 con 0x7fa904108510 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.019+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fa904195ef0 con 0x7fa8f0039560 2024-04-25T20:57:13.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.019+0000 7fa8fbfff700 1 -- 172.21.15.37:0/1355905080 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fa904195ef0 con 0x7fa8f0039560 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 msgr2=0x7fa8f003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 0x7fa8f003ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006fd0 tx=0x7fa8f4006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 msgr2=0x7fa90407db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec003f40 tx=0x7fa8ec011890 comp rx=0 tx=0).stop 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 shutdown_connections 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa8f0039560 0x7fa8f003ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.021+0000 7fa90953c700 1 --2- 172.21.15.37:0/1355905080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa904108510 0x7fa90407db30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.022+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 >> 172.21.15.37:0/1355905080 conn(0x7fa904103b70 msgr2=0x7fa9041048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.022+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 shutdown_connections 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:13.022+0000 7fa90953c700 1 -- 172.21.15.37:0/1355905080 wait complete. 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T20:57:13.683 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T20:57:13.684 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-04-25T20:57:13.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:13 smithi037 ceph-mon[29195]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:15.167 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:14 smithi037 ceph-mon[29195]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:15 smithi037 ceph-mon[29195]: Deploying cephadm binary to smithi037 2024-04-25T20:57:19.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-04-25T20:57:19.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc26a034700 1 Processor -- start 2024-04-25T20:57:19.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc26a034700 1 -- start start 2024-04-25T20:57:19.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc26a034700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:19.117 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc26a034700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc264107dd0 con 0x7fc2641073e0 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38166/0 (socket says 172.21.15.37:38166) 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc2637fe700 1 -- 172.21.15.37:0/4051778656 learned_addr learned my addr 172.21.15.37:0/4051778656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.076+0000 7fc2637fe700 1 -- 172.21.15.37:0/4051778656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2641085e0 con 0x7fc2641073e0 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc2637fe700 1 --2- 172.21.15.37:0/4051778656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098a0 tx=0x7fc254009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e35c8554116b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:19.118 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc2627fc700 1 -- 172.21.15.37:0/4051778656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254004160 con 0x7fc2641073e0 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc2627fc700 1 -- 172.21.15.37:0/4051778656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc254004300 con 0x7fc2641073e0 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc2627fc700 1 -- 172.21.15.37:0/4051778656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254011af0 con 0x7fc2641073e0 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc26a034700 1 -- 172.21.15.37:0/4051778656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 msgr2=0x7fc264107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc26a034700 1 --2- 172.21.15.37:0/4051778656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2540098a0 tx=0x7fc254009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc26a034700 1 -- 172.21.15.37:0/4051778656 shutdown_connections 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc26a034700 1 --2- 172.21.15.37:0/4051778656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc264107800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:19.119 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.077+0000 7fc26a034700 1 -- 172.21.15.37:0/4051778656 >> 172.21.15.37:0/4051778656 conn(0x7fc2641029c0 msgr2=0x7fc264104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 -- 172.21.15.37:0/4051778656 shutdown_connections 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 -- 172.21.15.37:0/4051778656 wait complete. 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 Processor -- start 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 -- start start 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:19.120 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.078+0000 7fc26a034700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc26410b990 con 0x7fc2641073e0 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc2637fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38172/0 (socket says 172.21.15.37:38172) 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc2637fe700 1 -- 172.21.15.37:0/1733915687 learned_addr learned my addr 172.21.15.37:0/1733915687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc2637fe700 1 -- 172.21.15.37:0/1733915687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26419cd20 con 0x7fc2641073e0 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc2637fe700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc254000c00 tx=0x7fc254011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:19.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254003b20 con 0x7fc2641073e0 2024-04-25T20:57:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc25401a8c0 con 0x7fc2641073e0 2024-04-25T20:57:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26419cfb0 con 0x7fc2641073e0 2024-04-25T20:57:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc254024940 con 0x7fc2641073e0 2024-04-25T20:57:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.079+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26419fca0 con 0x7fc2641073e0 2024-04-25T20:57:19.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.080+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc254022020 con 0x7fc2641073e0 2024-04-25T20:57:19.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.080+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2640640a0 con 0x7fc2641073e0 2024-04-25T20:57:19.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.080+0000 7fc260ff9700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 0x7fc24c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:19.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.081+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc25404d7b0 con 0x7fc2641073e0 2024-04-25T20:57:19.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.081+0000 7fc262ffd700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 0x7fc24c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:19.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.081+0000 7fc262ffd700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 0x7fc24c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.086+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc25402c3a0 con 0x7fc2641073e0 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:14.303+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7fc26402dff0 con 0x7fc24c039600 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.540+0000 7fc260ff9700 1 -- 172.21.15.37:0/1733915687 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc26402dff0 con 0x7fc24c039600 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 msgr2=0x7fc24c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 0x7fc24c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:19.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 msgr2=0x7fc26419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc254000c00 tx=0x7fc254011b70 comp rx=0 tx=0).stop 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 shutdown_connections 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fc24c039600 0x7fc24c03bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 --2- 172.21.15.37:0/1733915687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2641073e0 0x7fc26419c7e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.542+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 >> 172.21.15.37:0/1733915687 conn(0x7fc2641029c0 msgr2=0x7fc2641038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.543+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 shutdown_connections 2024-04-25T20:57:19.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:18.543+0000 7fc26a034700 1 -- 172.21.15.37:0/1733915687 wait complete. 2024-04-25T20:57:19.126 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-04-25T20:57:19.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:19 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:19.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:19 smithi037 ceph-mon[29195]: Added host smithi037 2024-04-25T20:57:19.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:19 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f97fe6700 1 Processor -- start 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f97fe6700 1 -- start start 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f97fe6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f97fe6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9007be60 con 0x7f2f90108fb0 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f95d82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f95d82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44102/0 (socket says 172.21.15.37:44102) 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.509+0000 7f2f95d82700 1 -- 172.21.15.37:0/3066214650 learned_addr learned my addr 172.21.15.37:0/3066214650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.510+0000 7f2f95d82700 1 -- 172.21.15.37:0/3066214650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9010b8e0 con 0x7f2f90108fb0 2024-04-25T20:57:20.328 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.510+0000 7f2f95d82700 1 --2- 172.21.15.37:0/3066214650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0098a0 tx=0x7f2f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abeef00cf80b181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.510+0000 7f2f94d80700 1 -- 172.21.15.37:0/3066214650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c004160 con 0x7f2f90108fb0 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.510+0000 7f2f94d80700 1 -- 172.21.15.37:0/3066214650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f8c004300 con 0x7f2f90108fb0 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.510+0000 7f2f94d80700 1 -- 172.21.15.37:0/3066214650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c011af0 con 0x7f2f90108fb0 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- 172.21.15.37:0/3066214650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 msgr2=0x7f2f9010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/3066214650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0098a0 tx=0x7f2f8c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- 172.21.15.37:0/3066214650 shutdown_connections 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/3066214650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9010b3a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- 172.21.15.37:0/3066214650 >> 172.21.15.37:0/3066214650 conn(0x7f2f901029c0 msgr2=0x7f2f90104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- 172.21.15.37:0/3066214650 shutdown_connections 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- 172.21.15.37:0/3066214650 wait complete. 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 Processor -- start 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- start start 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:20.329 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.511+0000 7f2f97fe6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010c850 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f95d82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f95d82700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44116/0 (socket says 172.21.15.37:44116) 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f95d82700 1 -- 172.21.15.37:0/2192946736 learned_addr learned my addr 172.21.15.37:0/2192946736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f95d82700 1 -- 172.21.15.37:0/2192946736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9019cc80 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f95d82700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c003f40 tx=0x7f2f8c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c0038b0 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f8c003a50 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.512+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9019cf10 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f8c01a5c0 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9019fc00 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2f8c01a870 con 0x7f2f90108fb0 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f86ffd700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 0x7f2f7c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:20.330 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f8c053060 con 0x7f2f90108fb0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.513+0000 7f2f95581700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 0x7f2f7c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.514+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f900640a0 con 0x7f2f90108fb0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.514+0000 7f2f95581700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 0x7f2f7c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.519+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f8c00be60 con 0x7f2f90108fb0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.738+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2f90062e80 con 0x7f2f7c0395b0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.743+0000 7f2f86ffd700 1 -- 172.21.15.37:0/2192946736 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2f90062e80 con 0x7f2f7c0395b0 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 msgr2=0x7f2f7c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 0x7f2f7c03ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006fd0 tx=0x7f2f80006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 msgr2=0x7f2f9019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c003f40 tx=0x7f2f8c011890 comp rx=0 tx=0).stop 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 shutdown_connections 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f2f7c0395b0 0x7f2f7c03ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 --2- 172.21.15.37:0/2192946736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f90108fb0 0x7f2f9019c740 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:20.331 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 >> 172.21.15.37:0/2192946736 conn(0x7f2f901029c0 msgr2=0x7f2f90103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:20.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 shutdown_connections 2024-04-25T20:57:20.332 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:19.745+0000 7f2f97fe6700 1 -- 172.21.15.37:0/2192946736 wait complete. 2024-04-25T20:57:20.332 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-04-25T20:57:20.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:20 smithi037 ceph-mon[29195]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:20.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:20 smithi037 ceph-mon[29195]: Saving service mon spec with placement count:5 2024-04-25T20:57:20.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:20 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:22.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.154+0000 7f3c11f8b700 1 Processor -- start 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.154+0000 7f3c11f8b700 1 -- start start 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.154+0000 7f3c11f8b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.154+0000 7f3c11f8b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c07be80 con 0x7f3c0c108f40 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44120/0 (socket says 172.21.15.37:44120) 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0b7fe700 1 -- 172.21.15.37:0/2849117935 learned_addr learned my addr 172.21.15.37:0/2849117935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0b7fe700 1 -- 172.21.15.37:0/2849117935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c10b870 con 0x7f3c0c108f40 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0b7fe700 1 --2- 172.21.15.37:0/2849117935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098a0 tx=0x7f3c00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8851b77138b3392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0a7fc700 1 -- 172.21.15.37:0/2849117935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00004160 con 0x7f3c0c108f40 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0a7fc700 1 -- 172.21.15.37:0/2849117935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c00004300 con 0x7f3c0c108f40 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.155+0000 7f3c0a7fc700 1 -- 172.21.15.37:0/2849117935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00011af0 con 0x7f3c0c108f40 2024-04-25T20:57:22.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 -- 172.21.15.37:0/2849117935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 msgr2=0x7f3c0c10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/2849117935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c000098a0 tx=0x7f3c00009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 -- 172.21.15.37:0/2849117935 shutdown_connections 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/2849117935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c10b330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 -- 172.21.15.37:0/2849117935 >> 172.21.15.37:0/2849117935 conn(0x7f3c0c1029b0 msgr2=0x7f3c0c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 -- 172.21.15.37:0/2849117935 shutdown_connections 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.156+0000 7f3c11f8b700 1 -- 172.21.15.37:0/2849117935 wait complete. 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c11f8b700 1 Processor -- start 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c11f8b700 1 -- start start 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c11f8b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c11f8b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c10c830 con 0x7f3c0c108f40 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c0b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44122/0 (socket says 172.21.15.37:44122) 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c0b7fe700 1 -- 172.21.15.37:0/1484489977 learned_addr learned my addr 172.21.15.37:0/1484489977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:22.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.157+0000 7f3c0b7fe700 1 -- 172.21.15.37:0/1484489977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c19c990 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c0b7fe700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009fb0 tx=0x7f3c00011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c000036a0 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c0001a430 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c19cc20 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c00019550 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.158+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c19f910 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.159+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c0640a0 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.159+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3c00019850 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.159+0000 7f3c08ff9700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 0x7f3bf403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.159+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3c0004dac0 con 0x7f3c0c108f40 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.159+0000 7f3c0affd700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 0x7f3bf403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:22.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.160+0000 7f3c0affd700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 0x7f3bf403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.169+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c0002d960 con 0x7f3c0c108f40 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.392+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3c0c103c00 con 0x7f3bf4039560 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.398+0000 7f3c08ff9700 1 -- 172.21.15.37:0/1484489977 <== mgr.14120 v2:172.21.15.37:6800/3898721946 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3c0c103c00 con 0x7f3bf4039560 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 msgr2=0x7f3bf403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 0x7f3bf403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc006fd0 tx=0x7f3bfc006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 msgr2=0x7f3c0c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009fb0 tx=0x7f3c00011940 comp rx=0 tx=0).stop 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 shutdown_connections 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f3bf4039560 0x7f3bf403ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 --2- 172.21.15.37:0/1484489977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c0c108f40 0x7f3c0c19c450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 >> 172.21.15.37:0/1484489977 conn(0x7f3c0c1029b0 msgr2=0x7f3c0c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 shutdown_connections 2024-04-25T20:57:22.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:21.401+0000 7f3c11f8b700 1 -- 172.21.15.37:0/1484489977 wait complete. 2024-04-25T20:57:22.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:22 smithi037 ceph-mon[29195]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:22.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:22 smithi037 ceph-mon[29195]: Saving service mgr spec with placement count:2 2024-04-25T20:57:22.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:22 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:22.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:22 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:23 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:23 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4189027407' entity='client.admin' 2024-04-25T20:57:24.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.783+0000 7fa4c9f72700 1 Processor -- start 2024-04-25T20:57:24.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.783+0000 7fa4c9f72700 1 -- start start 2024-04-25T20:57:24.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c9f72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c9f72700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c4108ed0 con 0x7fa4c41084e0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44138/0 (socket says 172.21.15.37:44138) 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c37fe700 1 -- 172.21.15.37:0/2523672044 learned_addr learned my addr 172.21.15.37:0/2523672044 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c37fe700 1 -- 172.21.15.37:0/2523672044 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c4109690 con 0x7fa4c41084e0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.784+0000 7fa4c37fe700 1 --2- 172.21.15.37:0/2523672044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0098a0 tx=0x7fa4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42be180a9143693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c27fc700 1 -- 172.21.15.37:0/2523672044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac004160 con 0x7fa4c41084e0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c27fc700 1 -- 172.21.15.37:0/2523672044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac004300 con 0x7fa4c41084e0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c27fc700 1 -- 172.21.15.37:0/2523672044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac011a20 con 0x7fa4c41084e0 2024-04-25T20:57:24.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 -- 172.21.15.37:0/2523672044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 msgr2=0x7fa4c4108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/2523672044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0098a0 tx=0x7fa4ac009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 -- 172.21.15.37:0/2523672044 shutdown_connections 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/2523672044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c4108900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 -- 172.21.15.37:0/2523672044 >> 172.21.15.37:0/2523672044 conn(0x7fa4c4103b40 msgr2=0x7fa4c4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 -- 172.21.15.37:0/2523672044 shutdown_connections 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.785+0000 7fa4c9f72700 1 -- 172.21.15.37:0/2523672044 wait complete. 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c9f72700 1 Processor -- start 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c9f72700 1 -- start start 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c9f72700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c9f72700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c410ca40 con 0x7fa4c41084e0 2024-04-25T20:57:24.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44140/0 (socket says 172.21.15.37:44140) 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.786+0000 7fa4c37fe700 1 -- 172.21.15.37:0/4189027407 learned_addr learned my addr 172.21.15.37:0/4189027407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c37fe700 1 -- 172.21.15.37:0/4189027407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c419eb30 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c37fe700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0119f0 tx=0x7fa4ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac0039e0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac01b8f0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c419edc0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4ac0237a0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.787+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c41a1ab0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.788+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa4ac02c3f0 con 0x7fa4c41084e0 2024-04-25T20:57:24.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.788+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c40640a0 con 0x7fa4c41084e0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.788+0000 7fa4c0ff9700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 0x7fa4b003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.788+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa4ac04e130 con 0x7fa4c41084e0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.788+0000 7fa4c2ffd700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 0x7fa4b003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.789+0000 7fa4c2ffd700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 0x7fa4b003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:22.793+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4ac01f030 con 0x7fa4c41084e0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.023+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fa4c41a2060 con 0x7fa4c41084e0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.026+0000 7fa4c0ff9700 1 -- 172.21.15.37:0/4189027407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fa4ac0202b0 con 0x7fa4c41084e0 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 msgr2=0x7fa4b003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 0x7fa4b003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4006fd0 tx=0x7fa4b4006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 msgr2=0x7fa4c419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0119f0 tx=0x7fa4ac011ad0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 shutdown_connections 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fa4b0039560 0x7fa4b003ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 --2- 172.21.15.37:0/4189027407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4c41084e0 0x7fa4c419e5f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 >> 172.21.15.37:0/4189027407 conn(0x7fa4c4103b40 msgr2=0x7fa4c4104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.029+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 shutdown_connections 2024-04-25T20:57:24.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:23.030+0000 7fa4c9f72700 1 -- 172.21.15.37:0/4189027407 wait complete. 2024-04-25T20:57:25.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:24 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:25.421 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.655+0000 7fd4f7dc2700 1 Processor -- start 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.655+0000 7fd4f7dc2700 1 -- start start 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.655+0000 7fd4f7dc2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.655+0000 7fd4f7dc2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0109400 con 0x7fd4f0108a10 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f5b5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f5b5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44152/0 (socket says 172.21.15.37:44152) 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f5b5e700 1 -- 172.21.15.37:0/3204549582 learned_addr learned my addr 172.21.15.37:0/3204549582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f5b5e700 1 -- 172.21.15.37:0/3204549582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f0107620 con 0x7fd4f0108a10 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f5b5e700 1 --2- 172.21.15.37:0/3204549582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0098a0 tx=0x7fd4ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b53f01e7a3fa7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f4b5c700 1 -- 172.21.15.37:0/3204549582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ec004160 con 0x7fd4f0108a10 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f4b5c700 1 -- 172.21.15.37:0/3204549582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec004300 con 0x7fd4f0108a10 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.656+0000 7fd4f4b5c700 1 -- 172.21.15.37:0/3204549582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ec011a20 con 0x7fd4f0108a10 2024-04-25T20:57:25.422 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/3204549582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 msgr2=0x7fd4f0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/3204549582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0098a0 tx=0x7fd4ec009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/3204549582 shutdown_connections 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/3204549582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f0108e30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/3204549582 >> 172.21.15.37:0/3204549582 conn(0x7fd4f01021b0 msgr2=0x7fd4f01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/3204549582 shutdown_connections 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.657+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/3204549582 wait complete. 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f7dc2700 1 Processor -- start 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f7dc2700 1 -- start start 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f7dc2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f7dc2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010b9e0 con 0x7fd4f0108a10 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f5b5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f5b5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44160/0 (socket says 172.21.15.37:44160) 2024-04-25T20:57:25.423 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f5b5e700 1 -- 172.21.15.37:0/1757918390 learned_addr learned my addr 172.21.15.37:0/1757918390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f5b5e700 1 -- 172.21.15.37:0/1757918390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f0192ae0 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4f5b5e700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0116e0 tx=0x7fd4ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ec0039e0 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.658+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec01b8f0 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.659+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f0190cd0 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.659+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4ec023760 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.659+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f0191210 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.659+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd4ec01b440 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.660+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f00640a0 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.660+0000 7fd4e6ffd700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 0x7fd4dc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.660+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd4ec04de50 con 0x7fd4f0108a10 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.660+0000 7fd4f535d700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 0x7fd4dc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.660+0000 7fd4f535d700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 0x7fd4dc03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:25.424 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.665+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4ec01f030 con 0x7fd4f0108a10 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.877+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd4f0196030 con 0x7fd4f0108a10 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.880+0000 7fd4e6ffd700 1 -- 172.21.15.37:0/1757918390 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd4ec04f020 con 0x7fd4f0108a10 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 msgr2=0x7fd4dc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 0x7fd4dc03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 msgr2=0x7fd4f01925a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0116e0 tx=0x7fd4ec011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 shutdown_connections 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7fd4dc0395b0 0x7fd4dc03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 --2- 172.21.15.37:0/1757918390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f0108a10 0x7fd4f01925a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 >> 172.21.15.37:0/1757918390 conn(0x7fd4f01021b0 msgr2=0x7fd4f0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 shutdown_connections 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:24.882+0000 7fd4f7dc2700 1 -- 172.21.15.37:0/1757918390 wait complete. 2024-04-25T20:57:25.425 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-04-25T20:57:25.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:25 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1757918390' entity='client.admin' 2024-04-25T20:57:27.125 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:26 smithi037 ceph-mon[29195]: from='mgr.14120 172.21.15.37:0/781206614' entity='mgr.x' 2024-04-25T20:57:27.125 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:26 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3449208341' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T20:57:27.125 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setuser ceph since I am not root 2024-04-25T20:57:27.126 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setgroup ceph since I am not root 2024-04-25T20:57:27.126 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:27.124+0000 7f9262347200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:57:27.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:27.315+0000 7f9262347200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:27.423 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:27.422+0000 7f9262347200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.828+0000 7f45d7bc6700 1 Processor -- start 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d7bc6700 1 -- start start 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d7bc6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d7bc6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45d0107210 con 0x7f45d0106820 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d5962700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d5962700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44170/0 (socket says 172.21.15.37:44170) 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d5962700 1 -- 172.21.15.37:0/639467775 learned_addr learned my addr 172.21.15.37:0/639467775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d5962700 1 -- 172.21.15.37:0/639467775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d01097b0 con 0x7f45d0106820 2024-04-25T20:57:27.598 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.829+0000 7f45d5962700 1 --2- 172.21.15.37:0/639467775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0098a0 tx=0x7f45cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dfa6216eac86ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d4960700 1 -- 172.21.15.37:0/639467775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc004160 con 0x7f45d0106820 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d4960700 1 -- 172.21.15.37:0/639467775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45cc004300 con 0x7f45d0106820 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d4960700 1 -- 172.21.15.37:0/639467775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc011a20 con 0x7f45d0106820 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 -- 172.21.15.37:0/639467775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 msgr2=0x7f45d0106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/639467775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0098a0 tx=0x7f45cc009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 -- 172.21.15.37:0/639467775 shutdown_connections 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/639467775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0106c40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 -- 172.21.15.37:0/639467775 >> 172.21.15.37:0/639467775 conn(0x7f45d01021b0 msgr2=0x7f45d01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 -- 172.21.15.37:0/639467775 shutdown_connections 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.830+0000 7f45d7bc6700 1 -- 172.21.15.37:0/639467775 wait complete. 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d7bc6700 1 Processor -- start 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d7bc6700 1 -- start start 2024-04-25T20:57:27.599 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d7bc6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d7bc6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45d010cb60 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d5962700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d5962700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44172/0 (socket says 172.21.15.37:44172) 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.831+0000 7f45d5962700 1 -- 172.21.15.37:0/3449208341 learned_addr learned my addr 172.21.15.37:0/3449208341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45d5962700 1 -- 172.21.15.37:0/3449208341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0109570 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45d5962700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009520 tx=0x7f45cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc0039e0 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45cc01b8f0 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc019770 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d0107760 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.832+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d0107ca0 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.833+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b4005360 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.833+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f45cc01b440 con 0x7f45d0106820 2024-04-25T20:57:27.600 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.834+0000 7f45c6ffd700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 0x7f45bc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.834+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45cc04deb0 con 0x7f45d0106820 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.834+0000 7f45d5161700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 0x7f45bc03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.834+0000 7f45d5161700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 0x7f45bc03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45c0006fd0 tx=0x7f45c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:25.839+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45cc01f030 con 0x7f45d0106820 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.115+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f45b4005650 con 0x7f45d0106820 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.938+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f45cc00b3e0 con 0x7f45d0106820 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.940+0000 7f45c6ffd700 1 -- 172.21.15.37:0/3449208341 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f45cc019c90 con 0x7f45d0106820 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 msgr2=0x7f45bc03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 0x7f45bc03bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f45c0006fd0 tx=0x7f45c0006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 msgr2=0x7f45d0109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009520 tx=0x7f45cc011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 shutdown_connections 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f45bc039600 0x7f45bc03bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.601 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 --2- 172.21.15.37:0/3449208341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d0106820 0x7f45d0109030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 >> 172.21.15.37:0/3449208341 conn(0x7f45d01021b0 msgr2=0x7f45d0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 shutdown_connections 2024-04-25T20:57:27.602 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:26.941+0000 7f45d7bc6700 1 -- 172.21.15.37:0/3449208341 wait complete. 2024-04-25T20:57:27.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3449208341' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T20:57:27.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-mon[29195]: mgrmap e9: x(active, since 20s) 2024-04-25T20:57:27.986 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:27.747+0000 7f9262347200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:57:27.986 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:27.835+0000 7f9262347200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:57:29.206 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:28 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3290743301' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:57:29.206 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:29.109+0000 7f9262347200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.385+0000 7f046a3e8700 1 Processor -- start 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.385+0000 7f046a3e8700 1 -- start start 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.385+0000 7f046a3e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.385+0000 7f046a3e8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f046407bec0 con 0x7f0464108fa0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0463fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0463fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44200/0 (socket says 172.21.15.37:44200) 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0463fff700 1 -- 172.21.15.37:0/3023440804 learned_addr learned my addr 172.21.15.37:0/3023440804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0463fff700 1 -- 172.21.15.37:0/3023440804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046410b8d0 con 0x7f0464108fa0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0463fff700 1 --2- 172.21.15.37:0/3023440804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69affac51a330099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0462ffd700 1 -- 172.21.15.37:0/3023440804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454004160 con 0x7f0464108fa0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0462ffd700 1 -- 172.21.15.37:0/3023440804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0454004300 con 0x7f0464108fa0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.386+0000 7f0462ffd700 1 -- 172.21.15.37:0/3023440804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454011a20 con 0x7f0464108fa0 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 -- 172.21.15.37:0/3023440804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 msgr2=0x7f046410b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3023440804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04540098a0 tx=0x7f0454009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 -- 172.21.15.37:0/3023440804 shutdown_connections 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3023440804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f046410b390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 -- 172.21.15.37:0/3023440804 >> 172.21.15.37:0/3023440804 conn(0x7f0464102a10 msgr2=0x7f0464104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 -- 172.21.15.37:0/3023440804 shutdown_connections 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.387+0000 7f046a3e8700 1 -- 172.21.15.37:0/3023440804 wait complete. 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f046a3e8700 1 Processor -- start 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f046a3e8700 1 -- start start 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f046a3e8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f046a3e8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f046410c8e0 con 0x7f0464108fa0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f0463fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f0463fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44208/0 (socket says 172.21.15.37:44208) 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f0463fff700 1 -- 172.21.15.37:0/3290743301 learned_addr learned my addr 172.21.15.37:0/3290743301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f0463fff700 1 -- 172.21.15.37:0/3290743301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0464108d60 con 0x7f0464108fa0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.388+0000 7f0463fff700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04540116e0 tx=0x7f0454011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.389+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0454003b20 con 0x7f0464108fa0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.389+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045401b8f0 con 0x7f0464108fa0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.389+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0464106f50 con 0x7f0464108fa0 2024-04-25T20:57:29.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.389+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04540238c0 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.389+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0464107490 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04640640a0 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f0454023a60 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f04617fa700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 0x7f044c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f045404e060 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f04637fe700 1 -- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 msgr2=0x7f044c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.390+0000 7f04637fe700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 0x7f044c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.395+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f045401f030 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.591+0000 7f04637fe700 1 -- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 msgr2=0x7f044c03bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.591+0000 7f04637fe700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 0x7f044c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.663+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f04641077d0 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.663+0000 7f04617fa700 1 -- 172.21.15.37:0/3290743301 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f04641077d0 con 0x7f0464108fa0 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 msgr2=0x7f044c03bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:29.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 0x7f044c03bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 msgr2=0x7f0464108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f04540116e0 tx=0x7f0454011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 shutdown_connections 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f044c039610 0x7f044c03bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 --2- 172.21.15.37:0/3290743301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0464108fa0 0x7f0464108820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 >> 172.21.15.37:0/3290743301 conn(0x7f0464102a10 msgr2=0x7f04641034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 shutdown_connections 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:28.666+0000 7f046a3e8700 1 -- 172.21.15.37:0/3290743301 wait complete. 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-25T20:57:29.218 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-04-25T20:57:29.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:29.291+0000 7f9262347200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:29.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:29.470+0000 7f9262347200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:57:30.082 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:29.775+0000 7f9262347200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:57:30.083 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:29.871+0000 7f9262347200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:57:30.083 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:30.081+0000 7f9262347200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:57:30.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:30.282+0000 7f9262347200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:31.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:30.904+0000 7f9262347200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:57:31.669 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:31 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:31.358+0000 7f9262347200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:57:31.669 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:31 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:31.458+0000 7f9262347200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:57:31.669 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:31 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:31.668+0000 7f9262347200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:57:32.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.217+0000 7f9262347200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:57:32.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.313+0000 7f9262347200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:57:32.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.566+0000 7f9262347200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:57:32.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.704+0000 7f9262347200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:57:32.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.803+0000 7f9262347200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:57:32.925 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:32.924+0000 7f9262347200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:33.735 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:33.235+0000 7f9262347200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:57:34.147 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:33.864+0000 7f9262347200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:57:34.147 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T20:57:34.090+0000 7f9262347200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:57:34.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T20:57:34.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: Activating manager daemon x 2024-04-25T20:57:34.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T20:57:34.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: mgrmap e10: x(active, starting, since 0.00322873s) 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T20:57:34.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:34] ENGINE Bus STARTING 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:34] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:34] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: [25/Apr/2024:20:57:34] ENGINE Bus STARTED 2024-04-25T20:57:35.381 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:35 smithi037 ceph-mon[29195]: mgrmap e11: x(active, since 1.00692s) 2024-04-25T20:57:35.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:57:35.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T20:57:35.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:57:35.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:57:35.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.683+0000 7f4af96c2700 1 Processor -- start 2024-04-25T20:57:35.674 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.683+0000 7f4af96c2700 1 -- start start 2024-04-25T20:57:35.674 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.683+0000 7f4af96c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.674 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.683+0000 7f4af96c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4af407bec0 con 0x7f4af4108fa0 2024-04-25T20:57:35.674 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.674 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42714/0 (socket says 172.21.15.37:42714) 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af2ffd700 1 -- 172.21.15.37:0/1724551854 learned_addr learned my addr 172.21.15.37:0/1724551854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af2ffd700 1 -- 172.21.15.37:0/1724551854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af410b8d0 con 0x7f4af4108fa0 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af2ffd700 1 --2- 172.21.15.37:0/1724551854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098a0 tx=0x7f4ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4ad3edc2178047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af1ffb700 1 -- 172.21.15.37:0/1724551854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae8004160 con 0x7f4af4108fa0 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af1ffb700 1 -- 172.21.15.37:0/1724551854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae8004300 con 0x7f4af4108fa0 2024-04-25T20:57:35.675 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.684+0000 7f4af1ffb700 1 -- 172.21.15.37:0/1724551854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae8011a20 con 0x7f4af4108fa0 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 -- 172.21.15.37:0/1724551854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 msgr2=0x7f4af410b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 --2- 172.21.15.37:0/1724551854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80098a0 tx=0x7f4ae8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 -- 172.21.15.37:0/1724551854 shutdown_connections 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 --2- 172.21.15.37:0/1724551854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af410b390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 -- 172.21.15.37:0/1724551854 >> 172.21.15.37:0/1724551854 conn(0x7f4af4102a10 msgr2=0x7f4af4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:35.676 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 -- 172.21.15.37:0/1724551854 shutdown_connections 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.685+0000 7f4af96c2700 1 -- 172.21.15.37:0/1724551854 wait complete. 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af96c2700 1 Processor -- start 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af96c2700 1 -- start start 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af96c2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af96c2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4af410c8e0 con 0x7f4af4108fa0 2024-04-25T20:57:35.677 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42716/0 (socket says 172.21.15.37:42716) 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.686+0000 7f4af2ffd700 1 -- 172.21.15.37:0/3654893087 learned_addr learned my addr 172.21.15.37:0/3654893087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4af2ffd700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af419eb30 con 0x7f4af4108fa0 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4af2ffd700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80116e0 tx=0x7f4ae8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae8003b20 con 0x7f4af4108fa0 2024-04-25T20:57:35.678 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af419edc0 con 0x7f4af4108fa0 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae801b8f0 con 0x7f4af4108fa0 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ae80238c0 con 0x7f4af4108fa0 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.687+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a1ab0 con 0x7f4af4108fa0 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4ae8023b70 con 0x7f4af4108fa0 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4adbfff700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.679 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4adc03c180 con 0x7f4adc0395b0 2024-04-25T20:57:35.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4af27fc700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:35.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4ae804e0b0 con 0x7f4af4108fa0 2024-04-25T20:57:35.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.688+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:35.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.889+0000 7f4af27fc700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:35.680 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:29.889+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:30.290+0000 7f4af27fc700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:30.290+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:31.091+0000 7f4af27fc700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:31.091+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:32.692+0000 7f4af27fc700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/3898721946 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:32.692+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:57:35.681 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:34.093+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f4ae801baa0 con 0x7f4af4108fa0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:34.094+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 msgr2=0x7f4adc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:34.094+0000 7f4adbfff700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.095+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f4ae8050230 con 0x7f4af4108fa0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.095+0000 7f4adbfff700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 0x7f4adc03f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.096+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4ae800be90 con 0x7f4adc03cfe0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.096+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 0x7f4adc03f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.097+0000 7f4af27fc700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 0x7f4adc03f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4003a10 tx=0x7f4ae40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.097+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4ae800be90 con 0x7f4adc03cfe0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.104+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4af4069b90 con 0x7f4adc03cfe0 2024-04-25T20:57:35.682 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.104+0000 7f4adbfff700 1 -- 172.21.15.37:0/3654893087 <== mgr.14152 v2:172.21.15.37:6800/2407888079 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4af4069b90 con 0x7f4adc03cfe0 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 msgr2=0x7f4adc03f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 0x7f4adc03f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4003a10 tx=0x7f4ae40092b0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 msgr2=0x7f4af419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80116e0 tx=0x7f4ae8011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 shutdown_connections 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4adc03cfe0 0x7f4adc03f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af4108fa0 0x7f4af419e5f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 --2- 172.21.15.37:0/3654893087 >> [v2:172.21.15.37:6800/3898721946,v1:172.21.15.37:6801/3898721946] conn(0x7f4adc0395b0 0x7f4adc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 >> 172.21.15.37:0/3654893087 conn(0x7f4af4102a10 msgr2=0x7f4af4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 shutdown_connections 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:35.105+0000 7f4af96c2700 1 -- 172.21.15.37:0/3654893087 wait complete. 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-04-25T20:57:35.683 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-04-25T20:57:36.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:36 smithi037 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:57:36.351 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:36 smithi037 ceph-mon[29195]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:57:37.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:37 smithi037 ceph-mon[29195]: mgrmap e12: x(active, since 2s) 2024-04-25T20:57:37.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:37.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:37 smithi037 ceph-mon[29195]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:37.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:37.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.376+0000 7fcaa32ac700 1 Processor -- start 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.376+0000 7fcaa32ac700 1 -- start start 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.376+0000 7fcaa32ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.376+0000 7fcaa32ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c07be20 con 0x7fca9c108fa0 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42786/0 (socket says 172.21.15.37:42786) 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fcaa1048700 1 -- 172.21.15.37:0/1899399237 learned_addr learned my addr 172.21.15.37:0/1899399237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fcaa1048700 1 -- 172.21.15.37:0/1899399237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c10b8d0 con 0x7fca9c108fa0 2024-04-25T20:57:37.338 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fcaa1048700 1 --2- 172.21.15.37:0/1899399237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca980098a0 tx=0x7fca98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e06b0395a31625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fca93fff700 1 -- 172.21.15.37:0/1899399237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98004160 con 0x7fca9c108fa0 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fca93fff700 1 -- 172.21.15.37:0/1899399237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98004300 con 0x7fca9c108fa0 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.377+0000 7fca93fff700 1 -- 172.21.15.37:0/1899399237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98011a20 con 0x7fca9c108fa0 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 -- 172.21.15.37:0/1899399237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 msgr2=0x7fca9c10b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/1899399237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca980098a0 tx=0x7fca98009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 -- 172.21.15.37:0/1899399237 shutdown_connections 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/1899399237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c10b390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 -- 172.21.15.37:0/1899399237 >> 172.21.15.37:0/1899399237 conn(0x7fca9c1029b0 msgr2=0x7fca9c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 -- 172.21.15.37:0/1899399237 shutdown_connections 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.378+0000 7fcaa32ac700 1 -- 172.21.15.37:0/1899399237 wait complete. 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.379+0000 7fcaa32ac700 1 Processor -- start 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.379+0000 7fcaa32ac700 1 -- start start 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.379+0000 7fcaa32ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.379+0000 7fcaa32ac700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca9c10c870 con 0x7fca9c108fa0 2024-04-25T20:57:37.339 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa1048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42794/0 (socket says 172.21.15.37:42794) 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa1048700 1 -- 172.21.15.37:0/2520582298 learned_addr learned my addr 172.21.15.37:0/2520582298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa1048700 1 -- 172.21.15.37:0/2520582298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9c19c960 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa1048700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca98004660 tx=0x7fca98011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98003aa0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9c19cbf0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca9c19f8e0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9801b8f0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.380+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca98023900 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.381+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca9c0640a0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.381+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fca98023bc0 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.382+0000 7fca927fc700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 0x7fca8803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.382+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fca9804e160 con 0x7fca9c108fa0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.382+0000 7fcaa0847700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 0x7fca8803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.382+0000 7fcaa0847700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 0x7fca8803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:37.340 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.387+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca9801f030 con 0x7fca9c108fa0 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.608+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fca9c103c20 con 0x7fca88039600 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.681+0000 7fca927fc700 1 -- 172.21.15.37:0/2520582298 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fca9c103c20 con 0x7fca88039600 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 msgr2=0x7fca8803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 0x7fca8803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 msgr2=0x7fca9c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca98004660 tx=0x7fca98011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 shutdown_connections 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fca88039600 0x7fca8803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 --2- 172.21.15.37:0/2520582298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca9c108fa0 0x7fca9c19c420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 >> 172.21.15.37:0/2520582298 conn(0x7fca9c1029b0 msgr2=0x7fca9c103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 shutdown_connections 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:36.683+0000 7fcaa32ac700 1 -- 172.21.15.37:0/2520582298 wait complete. 2024-04-25T20:57:37.341 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-04-25T20:57:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:38 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:38 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:39.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8Rvxdkt1DPL6qqLKnWsSbuvj5gg/BY5oOFrs5ckGYTNz4DOCHKe7C", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714078658, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T20:57:39.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279f100700 1 Processor -- start 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279f100700 1 -- start start 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279f100700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279f100700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2798107dd0 con 0x7f27981073e0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279ce9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279ce9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42802/0 (socket says 172.21.15.37:42802) 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279ce9c700 1 -- 172.21.15.37:0/3088696613 learned_addr learned my addr 172.21.15.37:0/3088696613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.810+0000 7f279ce9c700 1 -- 172.21.15.37:0/3088696613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981085e0 con 0x7f27981073e0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.811+0000 7f279ce9c700 1 --2- 172.21.15.37:0/3088696613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f278c0098a0 tx=0x7f278c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1514fbb654df7d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.811+0000 7f27977fe700 1 -- 172.21.15.37:0/3088696613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c004160 con 0x7f27981073e0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.811+0000 7f27977fe700 1 -- 172.21.15.37:0/3088696613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c004300 con 0x7f27981073e0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.811+0000 7f27977fe700 1 -- 172.21.15.37:0/3088696613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c011a20 con 0x7f27981073e0 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- 172.21.15.37:0/3088696613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 msgr2=0x7f2798107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 --2- 172.21.15.37:0/3088696613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f278c0098a0 tx=0x7f278c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- 172.21.15.37:0/3088696613 shutdown_connections 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 --2- 172.21.15.37:0/3088696613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f2798107800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- 172.21.15.37:0/3088696613 >> 172.21.15.37:0/3088696613 conn(0x7f27981029c0 msgr2=0x7f2798104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- 172.21.15.37:0/3088696613 shutdown_connections 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- 172.21.15.37:0/3088696613 wait complete. 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 Processor -- start 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 -- start start 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.812+0000 7f279f100700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279f100700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f279810b990 con 0x7f27981073e0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279ce9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279ce9c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42806/0 (socket says 172.21.15.37:42806) 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279ce9c700 1 -- 172.21.15.37:0/872203859 learned_addr learned my addr 172.21.15.37:0/872203859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279ce9c700 1 -- 172.21.15.37:0/872203859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279819ccd0 con 0x7f27981073e0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279ce9c700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f278c0116e0 tx=0x7f278c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c0039e0 con 0x7f27981073e0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c01b8f0 con 0x7f27981073e0 2024-04-25T20:57:39.212 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279819cf60 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.813+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f278c023760 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.814+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279819fc00 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.814+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f278c023900 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.815+0000 7f2795ffb700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 0x7f278803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.815+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f278c04dfe0 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.815+0000 7f2797fff700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 0x7f278803bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.815+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27980640a0 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.815+0000 7f2797fff700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 0x7f278803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2784006fd0 tx=0x7f2784006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:37.820+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f278c01f030 con 0x7f27981073e0 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.048+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2798068ba0 con 0x7f2788039610 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.298+0000 7f2795ffb700 1 -- 172.21.15.37:0/872203859 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f278400a040 con 0x7f2788039610 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 msgr2=0x7f278803bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 0x7f278803bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2784006fd0 tx=0x7f2784006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 msgr2=0x7f279819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f278c0116e0 tx=0x7f278c011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 shutdown_connections 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2788039610 0x7f278803bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 --2- 172.21.15.37:0/872203859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27981073e0 0x7f279819c790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 >> 172.21.15.37:0/872203859 conn(0x7f27981029c0 msgr2=0x7f2798103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 shutdown_connections 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:38.300+0000 7f279f100700 1 -- 172.21.15.37:0/872203859 wait complete. 2024-04-25T20:57:39.214 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-04-25T20:57:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:39 smithi037 ceph-mon[29195]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:40.892 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:40 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2981253601' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f409b945700 1 Processor -- start 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f409b945700 1 -- start start 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f409b945700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f409b945700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f409407be80 con 0x7f4094108fc0 2024-04-25T20:57:40.900 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f40996e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f40996e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57858/0 (socket says 172.21.15.37:57858) 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f40996e1700 1 -- 172.21.15.37:0/2609512919 learned_addr learned my addr 172.21.15.37:0/2609512919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.051+0000 7f40996e1700 1 -- 172.21.15.37:0/2609512919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409410b8f0 con 0x7f4094108fc0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f40996e1700 1 --2- 172.21.15.37:0/2609512919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40900098a0 tx=0x7f4090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3c033c6904bae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f408bfff700 1 -- 172.21.15.37:0/2609512919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090004160 con 0x7f4094108fc0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f408bfff700 1 -- 172.21.15.37:0/2609512919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4090004300 con 0x7f4094108fc0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f408bfff700 1 -- 172.21.15.37:0/2609512919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090011a20 con 0x7f4094108fc0 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f409b945700 1 -- 172.21.15.37:0/2609512919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 msgr2=0x7f409410b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f409b945700 1 --2- 172.21.15.37:0/2609512919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f40900098a0 tx=0x7f4090009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f409b945700 1 -- 172.21.15.37:0/2609512919 shutdown_connections 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f409b945700 1 --2- 172.21.15.37:0/2609512919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409410b3b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.052+0000 7f409b945700 1 -- 172.21.15.37:0/2609512919 >> 172.21.15.37:0/2609512919 conn(0x7f4094102a70 msgr2=0x7f4094104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 -- 172.21.15.37:0/2609512919 shutdown_connections 2024-04-25T20:57:40.901 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 -- 172.21.15.37:0/2609512919 wait complete. 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 Processor -- start 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 -- start start 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f409b945700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f409410c8f0 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f40996e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f40996e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57862/0 (socket says 172.21.15.37:57862) 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.053+0000 7f40996e1700 1 -- 172.21.15.37:0/2981253601 learned_addr learned my addr 172.21.15.37:0/2981253601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f40996e1700 1 -- 172.21.15.37:0/2981253601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419cc30 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f40996e1700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40900116e0 tx=0x7f4090011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090003a60 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409001b8f0 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409419cec0 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4090019860 con 0x7f4094108fc0 2024-04-25T20:57:40.902 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.054+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409419fb60 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.055+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4090019b80 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.055+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40940640a0 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.055+0000 7f408a7fc700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 0x7f408003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.055+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f409004df60 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.055+0000 7f4098ee0700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 0x7f408003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.056+0000 7f4098ee0700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 0x7f408003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4084006fd0 tx=0x7f4084006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.060+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f409001f030 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.267+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f409402df80 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.268+0000 7f408a7fc700 1 -- 172.21.15.37:0/2981253601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f409001bda0 con 0x7f4094108fc0 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 msgr2=0x7f408003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 0x7f408003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4084006fd0 tx=0x7f4084006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 msgr2=0x7f409419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40900116e0 tx=0x7f4090011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.903 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 shutdown_connections 2024-04-25T20:57:40.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40800395b0 0x7f408003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 --2- 172.21.15.37:0/2981253601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4094108fc0 0x7f409419c6f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 >> 172.21.15.37:0/2981253601 conn(0x7f4094102a70 msgr2=0x7f4094103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 shutdown_connections 2024-04-25T20:57:40.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:40.270+0000 7f409b945700 1 -- 172.21.15.37:0/2981253601 wait complete. 2024-04-25T20:57:40.907 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:57:40.907 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-25T20:57:40.915 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:57:40.915 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-25T20:57:40.915 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-25T20:57:40.915 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T20:57:40.916 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037:8443/ 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout: Password: kfpmawl9l7 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config directory 2024-04-25T20:57:40.917 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-04-25T20:57:41.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:41 smithi037 ceph-mon[29195]: mgrmap e13: x(active, since 6s) 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.325+0000 7fcd9ccd0700 1 Processor -- start 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.325+0000 7fcd9ccd0700 1 -- start start 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.325+0000 7fcd9ccd0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.325+0000 7fcd9ccd0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9807be60 con 0x7fcd98106dd0 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57872/0 (socket says 172.21.15.37:57872) 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9659c700 1 -- 172.21.15.37:0/3223121086 learned_addr learned my addr 172.21.15.37:0/3223121086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:42.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9659c700 1 -- 172.21.15.37:0/3223121086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd98107730 con 0x7fcd98106dd0 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9659c700 1 --2- 172.21.15.37:0/3223121086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd880098a0 tx=0x7fcd88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26885df76b817ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9559a700 1 -- 172.21.15.37:0/3223121086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88004160 con 0x7fcd98106dd0 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9559a700 1 -- 172.21.15.37:0/3223121086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd88004300 con 0x7fcd98106dd0 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.326+0000 7fcd9559a700 1 -- 172.21.15.37:0/3223121086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88011a20 con 0x7fcd98106dd0 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/3223121086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 msgr2=0x7fcd981071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/3223121086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd880098a0 tx=0x7fcd88009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/3223121086 shutdown_connections 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/3223121086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd981071f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/3223121086 >> 172.21.15.37:0/3223121086 conn(0x7fcd98102a70 msgr2=0x7fcd98104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/3223121086 shutdown_connections 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.327+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/3223121086 wait complete. 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9ccd0700 1 Processor -- start 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9ccd0700 1 -- start start 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9ccd0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:42.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9ccd0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981087a0 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57876/0 (socket says 172.21.15.37:57876) 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9659c700 1 -- 172.21.15.37:0/1370554374 learned_addr learned my addr 172.21.15.37:0/1370554374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9659c700 1 -- 172.21.15.37:0/1370554374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9819ee10 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.328+0000 7fcd9659c700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd88000c00 tx=0x7fcd88011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.329+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88003aa0 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.329+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9819f0a0 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.329+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd8801b8f0 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.329+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd88023870 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.329+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd981a1d90 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcd88023b70 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd980640a0 con 0x7fcd98106dd0 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd877fe700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 0x7fcd8003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:42.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd8804e220 con 0x7fcd98106dd0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd95d9b700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 0x7fcd8003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.330+0000 7fcd95d9b700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 0x7fcd8003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c006fd0 tx=0x7fcd8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.335+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd8801f030 con 0x7fcd98106dd0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.545+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fcd9802dff0 con 0x7fcd98106dd0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.546+0000 7fcd877fe700 1 -- 172.21.15.37:0/1370554374 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fcd88018c10 con 0x7fcd98106dd0 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 msgr2=0x7fcd8003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 0x7fcd8003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c006fd0 tx=0x7fcd8c006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 msgr2=0x7fcd9819e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd88000c00 tx=0x7fcd88011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 shutdown_connections 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcd800396a0 0x7fcd8003bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 --2- 172.21.15.37:0/1370554374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd98106dd0 0x7fcd9819e8d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 >> 172.21.15.37:0/1370554374 conn(0x7fcd98102a70 msgr2=0x7fcd981951e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 shutdown_connections 2024-04-25T20:57:42.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:41.548+0000 7fcd9ccd0700 1 -- 172.21.15.37:0/1370554374 wait complete. 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.561+0000 7effcec5f700 1 Processor -- start 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.561+0000 7effcec5f700 1 -- start start 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.561+0000 7effcec5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.561+0000 7effcec5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effc8108ec0 con 0x7effc81084d0 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effcc9fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effcc9fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57886/0 (socket says 172.21.15.37:57886) 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effcc9fb700 1 -- 172.21.15.37:0/1687416619 learned_addr learned my addr 172.21.15.37:0/1687416619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effcc9fb700 1 -- 172.21.15.37:0/1687416619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc8109680 con 0x7effc81084d0 2024-04-25T20:57:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effcc9fb700 1 --2- 172.21.15.37:0/1687416619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effb80098a0 tx=0x7effb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be145608d9ea49c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effc77fe700 1 -- 172.21.15.37:0/1687416619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8004160 con 0x7effc81084d0 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effc77fe700 1 -- 172.21.15.37:0/1687416619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb8004300 con 0x7effc81084d0 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.562+0000 7effc77fe700 1 -- 172.21.15.37:0/1687416619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8011a20 con 0x7effc81084d0 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 -- 172.21.15.37:0/1687416619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 msgr2=0x7effc81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 --2- 172.21.15.37:0/1687416619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7effb80098a0 tx=0x7effb8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 -- 172.21.15.37:0/1687416619 shutdown_connections 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 --2- 172.21.15.37:0/1687416619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc81088f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 -- 172.21.15.37:0/1687416619 >> 172.21.15.37:0/1687416619 conn(0x7effc8103b10 msgr2=0x7effc8105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 -- 172.21.15.37:0/1687416619 shutdown_connections 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.563+0000 7effcec5f700 1 -- 172.21.15.37:0/1687416619 wait complete. 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcec5f700 1 Processor -- start 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcec5f700 1 -- start start 2024-04-25T20:57:43.430 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcec5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcec5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effc810ca30 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcc9fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcc9fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57888/0 (socket says 172.21.15.37:57888) 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcc9fb700 1 -- 172.21.15.37:0/1594869096 learned_addr learned my addr 172.21.15.37:0/1594869096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcc9fb700 1 -- 172.21.15.37:0/1594869096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc819c910 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcc9fb700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effb80116e0 tx=0x7effb8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb80039e0 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb801b8f0 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.564+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc819cba0 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.565+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effb8023760 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.565+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc819f890 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.565+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7effb8023900 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.566+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc80640a0 con 0x7effc81084d0 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.566+0000 7effc5ffb700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 0x7effb003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.431 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.566+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7effb804e140 con 0x7effc81084d0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.566+0000 7effc7fff700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 0x7effb003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.566+0000 7effc7fff700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 0x7effb003bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.571+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effb801f030 con 0x7effc81084d0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.895+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7effc819fe40 con 0x7effc81084d0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.897+0000 7effc5ffb700 1 -- 172.21.15.37:0/1594869096 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7effb8018bf0 con 0x7effc81084d0 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.899+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 msgr2=0x7effb003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.899+0000 7effcec5f700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 0x7effb003bb70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effbc006fd0 tx=0x7effbc006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.899+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 msgr2=0x7effc819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.899+0000 7effcec5f700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effb80116e0 tx=0x7effb8011aa0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 shutdown_connections 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7effb00396b0 0x7effb003bb70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 --2- 172.21.15.37:0/1594869096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effc81084d0 0x7effc819c3d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 >> 172.21.15.37:0/1594869096 conn(0x7effc8103b10 msgr2=0x7effc81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:43.432 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 shutdown_connections 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-25T20:57:42.900+0000 7effcec5f700 1 -- 172.21.15.37:0/1594869096 wait complete. 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.433 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-04-25T20:57:43.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.434 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T20:57:43.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:43.434 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-04-25T20:57:43.466 INFO:tasks.cephadm:Fetching config... 2024-04-25T20:57:43.466 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:57:43.467 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T20:57:43.482 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T20:57:43.483 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:57:43.483 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T20:57:43.539 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T20:57:43.539 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:57:43.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T20:57:43.605 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T20:57:43.606 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:57:43.606 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T20:57:43.663 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T20:57:43.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:57:43.737 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:57:43.748 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:57:43.789 INFO:teuthology.orchestra.run.smithi098.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:57:43.800 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:57:43.840 INFO:teuthology.orchestra.run.smithi177.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDRUzllGvQ9D8xHJCTGZBSWpUR7oFhG71c4Y6nTiG1n54R2Txrpl8Lk3Y0e/mfTqwtcl9OV4ngrRXxldElU1rcuyIC5G2PiJNvgv2xs5lXAbUsfK38kbCovYqKXChlrFhE005MfHT5p5f6mpFlVzY3vS+hbdqel3pTlyxTajfBOAJQyJIsS9Q4zRmZM7SVTm8lKiUpzUtrdlLoBgHb3HIvNQrYK5gKMA+wm+FxSAuRc/X9kouJWdITmcZrSfrkr62yq9Kx3GOeIy0JNAgkpq2xw5mxKm+HfPDFS3vqvrOaQNcD7SceETafLOCQCxB9uAl3/0JqDoXtNIneREUWTHnT4l2L3QdMcSYaG/kizJlBG9a/kxUNh1iC6c28i/T3CV+GXXFrY59ENKrymHXJOnnwccTV1gjP6NlA+DUndLCAjPNaxIHR/GrU1ApKgjkL2bUDj7xCkhzGSoxcMQFhqVocAFyGmIfPcMQQp/ttLNVZWZ/PqLI/1NZtw1QLq1iVYOcM= ceph-1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:57:43.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T20:57:43.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:43 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1594869096' entity='client.admin' 2024-04-25T20:57:44.102 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 -- 172.21.15.37:0/3089533980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 msgr2=0x7fb3a01017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 --2- 172.21.15.37:0/3089533980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01017c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009910 tx=0x7fb39c009c20 comp rx=0 tx=0).stop 2024-04-25T20:57:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 -- 172.21.15.37:0/3089533980 shutdown_connections 2024-04-25T20:57:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 --2- 172.21.15.37:0/3089533980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01017c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 -- 172.21.15.37:0/3089533980 >> 172.21.15.37:0/3089533980 conn(0x7fb3a00fc940 msgr2=0x7fb3a00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 -- 172.21.15.37:0/3089533980 shutdown_connections 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.497+0000 7fb3a849e700 1 -- 172.21.15.37:0/3089533980 wait complete. 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a849e700 1 Processor -- start 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a849e700 1 -- start start 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a849e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a849e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a01072e0 con 0x7fb3a01013a0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a623a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a623a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57910/0 (socket says 172.21.15.37:57910) 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a623a700 1 -- 172.21.15.37:0/169103177 learned_addr learned my addr 172.21.15.37:0/169103177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.498+0000 7fb3a623a700 1 -- 172.21.15.37:0/169103177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a0196f20 con 0x7fb3a01013a0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3a623a700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005950 tx=0x7fb39c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c026070 con 0x7fb3a01013a0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39c00bc00 con 0x7fb3a01013a0 2024-04-25T20:57:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a01971b0 con 0x7fb3a01013a0 2024-04-25T20:57:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c003a10 con 0x7fb3a01013a0 2024-04-25T20:57:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.499+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a0199ea0 con 0x7fb3a01013a0 2024-04-25T20:57:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.500+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb39c029070 con 0x7fb3a01013a0 2024-04-25T20:57:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.500+0000 7fb3977fe700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 0x7fb38c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.500+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb39c01e070 con 0x7fb3a01013a0 2024-04-25T20:57:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.500+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a0107a60 con 0x7fb3a01013a0 2024-04-25T20:57:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.500+0000 7fb3a5a39700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 0x7fb38c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.501+0000 7fb3a5a39700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 0x7fb38c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb390006fd0 tx=0x7fb390006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.506+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb39c030020 con 0x7fb3a01013a0 2024-04-25T20:57:46.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.715+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb3a002df80 con 0x7fb3a01013a0 2024-04-25T20:57:46.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.718+0000 7fb3977fe700 1 -- 172.21.15.37:0/169103177 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb39c013070 con 0x7fb3a01013a0 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 msgr2=0x7fb38c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 0x7fb38c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb390006fd0 tx=0x7fb390006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 msgr2=0x7fb3a01969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39c005950 tx=0x7fb39c004fa0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 shutdown_connections 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb38c0396b0 0x7fb38c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 --2- 172.21.15.37:0/169103177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a01013a0 0x7fb3a01969e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.721+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 >> 172.21.15.37:0/169103177 conn(0x7fb3a00fc940 msgr2=0x7fb3a00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.722+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 shutdown_connections 2024-04-25T20:57:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:46.722+0000 7fb3a849e700 1 -- 172.21.15.37:0/169103177 wait complete. 2024-04-25T20:57:47.286 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T20:57:47.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T20:57:47.814 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:47 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/169103177' entity='client.admin' 2024-04-25T20:57:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:49.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:49.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 -- 172.21.15.37:0/233294871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 msgr2=0x7fe5ec1017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 --2- 172.21.15.37:0/233294871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec1017e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009910 tx=0x7fe5e8009c20 comp rx=0 tx=0).stop 2024-04-25T20:57:49.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 -- 172.21.15.37:0/233294871 shutdown_connections 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 --2- 172.21.15.37:0/233294871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec1017e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 -- 172.21.15.37:0/233294871 >> 172.21.15.37:0/233294871 conn(0x7fe5ec0fc980 msgr2=0x7fe5ec0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 -- 172.21.15.37:0/233294871 shutdown_connections 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.390+0000 7fe5f436a700 1 -- 172.21.15.37:0/233294871 wait complete. 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f436a700 1 Processor -- start 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f436a700 1 -- start start 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f436a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f436a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec107300 con 0x7fe5ec1013c0 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f2106700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f2106700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40506/0 (socket says 172.21.15.37:40506) 2024-04-25T20:57:49.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f2106700 1 -- 172.21.15.37:0/1411311378 learned_addr learned my addr 172.21.15.37:0/1411311378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:49.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.391+0000 7fe5f2106700 1 -- 172.21.15.37:0/1411311378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec196fa0 con 0x7fe5ec1013c0 2024-04-25T20:57:49.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5f2106700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8004fb0 tx=0x7fe5e8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e8027070 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e800be10 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec197230 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5e8022de0 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.392+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec199f20 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe5e802a070 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ec107a80 con 0x7fe5ec1013c0 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5e37fe700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 0x7fe5d803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe5e801e030 con 0x7fe5ec1013c0 2024-04-25T20:57:49.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5f1905700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 0x7fe5d803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.393+0000 7fe5f1905700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 0x7fe5d803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:49.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.398+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5e8013070 con 0x7fe5ec1013c0 2024-04-25T20:57:49.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.610+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe5ec02e120 con 0x7fe5d8039650 2024-04-25T20:57:49.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.613+0000 7fe5e37fe700 1 -- 172.21.15.37:0/1411311378 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe5ec02e120 con 0x7fe5d8039650 2024-04-25T20:57:49.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 msgr2=0x7fe5d803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 0x7fe5d803bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 msgr2=0x7fe5ec196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8004fb0 tx=0x7fe5e8009f60 comp rx=0 tx=0).stop 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 shutdown_connections 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5d8039650 0x7fe5d803bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 --2- 172.21.15.37:0/1411311378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5ec1013c0 0x7fe5ec196a60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.615+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 >> 172.21.15.37:0/1411311378 conn(0x7fe5ec0fc980 msgr2=0x7fe5ec0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.616+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 shutdown_connections 2024-04-25T20:57:49.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:49.616+0000 7fe5f436a700 1 -- 172.21.15.37:0/1411311378 wait complete. 2024-04-25T20:57:50.271 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi098 2024-04-25T20:57:50.272 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:57:50.272 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:57:50.289 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:57:50.290 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:50.347 INFO:tasks.cephadm:Adding host smithi098 to orchestrator... 2024-04-25T20:57:50.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch host add smithi098 2024-04-25T20:57:50.598 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:50.865 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:50.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:51.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:51 smithi037 ceph-mon[29195]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:51.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:51 smithi037 ceph-mon[29195]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/2644951122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 msgr2=0x7f4ccc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/2644951122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc1019d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009910 tx=0x7f4cbc009c20 comp rx=0 tx=0).stop 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/2644951122 shutdown_connections 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/2644951122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc1019d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/2644951122 >> 172.21.15.37:0/2644951122 conn(0x7f4ccc076ea0 msgr2=0x7f4ccc0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/2644951122 shutdown_connections 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.011+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/2644951122 wait complete. 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4cd1e2c700 1 Processor -- start 2024-04-25T20:57:52.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4cd1e2c700 1 -- start start 2024-04-25T20:57:52.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4cd1e2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:52.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4cd1e2c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ccc10b820 con 0x7f4ccc1015f0 2024-04-25T20:57:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4ccb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4ccb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40528/0 (socket says 172.21.15.37:40528) 2024-04-25T20:57:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4ccb7fe700 1 -- 172.21.15.37:0/896072117 learned_addr learned my addr 172.21.15.37:0/896072117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:52.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.012+0000 7f4ccb7fe700 1 -- 172.21.15.37:0/896072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ccc1991d0 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4ccb7fe700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0056d0 tx=0x7f4cbc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cbc026070 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cbc00bba0 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ccc1994c0 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cbc003920 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.013+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ccc199980 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4cbc029070 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ccc102a80 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4cc8ff9700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 0x7f4cb803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4cbc01e070 con 0x7f4ccc1015f0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4ccaffd700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 0x7f4cb803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:52.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.014+0000 7f4ccaffd700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 0x7f4cb803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006fd0 tx=0x7f4cb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:52.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.019+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cbc013070 con 0x7f4ccc1015f0 2024-04-25T20:57:52.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:52.232+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}) -- 0x7f4ccc06a4b0 con 0x7f4cb8039650 2024-04-25T20:57:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:53 smithi037 ceph-mon[29195]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi098", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:53 smithi037 ceph-mon[29195]: Deploying cephadm binary to smithi098 2024-04-25T20:57:55.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:55 smithi037 ceph-mon[29195]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:57:56.442 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi098' with addr '172.21.15.98' 2024-04-25T20:57:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.440+0000 7f4cc8ff9700 1 -- 172.21.15.37:0/896072117 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4ccc06a4b0 con 0x7f4cb8039650 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 msgr2=0x7f4cb803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 0x7f4cb803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006fd0 tx=0x7f4cb4006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 msgr2=0x7f4ccc198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc0056d0 tx=0x7f4cbc0057b0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 shutdown_connections 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4cb8039650 0x7f4cb803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 --2- 172.21.15.37:0/896072117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ccc1015f0 0x7f4ccc198c90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.442+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 >> 172.21.15.37:0/896072117 conn(0x7f4ccc076ea0 msgr2=0x7f4ccc109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.443+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 shutdown_connections 2024-04-25T20:57:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:56.443+0000 7f4cd1e2c700 1 -- 172.21.15.37:0/896072117 wait complete. 2024-04-25T20:57:57.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:57:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:57.729 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:57 smithi037 ceph-mon[29195]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:57:57.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:57.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:57 smithi037 ceph-mon[29195]: Added host smithi098 2024-04-25T20:57:57.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:57.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:58.811 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:58.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 -- 172.21.15.37:0/4122238265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 msgr2=0x7f1ee8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/4122238265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8108930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009910 tx=0x7f1ed8009c20 comp rx=0 tx=0).stop 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 -- 172.21.15.37:0/4122238265 shutdown_connections 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/4122238265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8108930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 -- 172.21.15.37:0/4122238265 >> 172.21.15.37:0/4122238265 conn(0x7f1ee80fdf70 msgr2=0x7f1ee8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.840+0000 7f1eefcc2700 1 -- 172.21.15.37:0/4122238265 shutdown_connections 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eefcc2700 1 -- 172.21.15.37:0/4122238265 wait complete. 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eefcc2700 1 Processor -- start 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eefcc2700 1 -- start start 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eefcc2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eefcc2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810c9c0 con 0x7f1ee8108550 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eeda5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eeda5e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57848/0 (socket says 172.21.15.37:57848) 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.841+0000 7f1eeda5e700 1 -- 172.21.15.37:0/741262372 learned_addr learned my addr 172.21.15.37:0/741262372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1eeda5e700 1 -- 172.21.15.37:0/741262372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee8076be0 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1eeda5e700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80056d0 tx=0x7f1ed80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8026070 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed800bc00 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee8075280 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ed8003920 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.842+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee8075740 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.843+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1ed8029070 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.843+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee8103bc0 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.843+0000 7f1edeffd700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 0x7f1ed403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.843+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1ed801e070 con 0x7f1ee8108550 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.843+0000 7f1eed25d700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 0x7f1ed403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.844+0000 7f1eed25d700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 0x7f1ed403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006fd0 tx=0x7f1ee4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:58.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:58.848+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ed8013070 con 0x7f1ee8108550 2024-04-25T20:57:59.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.062+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ee8076270 con 0x7f1ed4039650 2024-04-25T20:57:59.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.063+0000 7f1edeffd700 1 -- 172.21.15.37:0/741262372 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f1ee8076270 con 0x7f1ed4039650 2024-04-25T20:57:59.064 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:57:59.064 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}] 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 msgr2=0x7f1ed403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 0x7f1ed403bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4006fd0 tx=0x7f1ee4006e40 comp rx=0 tx=0).stop 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 msgr2=0x7f1ee8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80056d0 tx=0x7f1ed80057b0 comp rx=0 tx=0).stop 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 shutdown_connections 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1ed4039650 0x7f1ed403bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 --2- 172.21.15.37:0/741262372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ee8108550 0x7f1ee8074d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 >> 172.21.15.37:0/741262372 conn(0x7f1ee80fdf70 msgr2=0x7f1ee80fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:59.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 shutdown_connections 2024-04-25T20:57:59.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:57:59.065+0000 7f1eefcc2700 1 -- 172.21.15.37:0/741262372 wait complete. 2024-04-25T20:57:59.617 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:59 smithi037 ceph-mon[29195]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:57:59.617 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:57:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:57:59.619 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi177 2024-04-25T20:57:59.619 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:57:59.619 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:57:59.637 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:57:59.637 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:59.695 INFO:tasks.cephadm:Adding host smithi177 to orchestrator... 2024-04-25T20:57:59.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch host add smithi177 2024-04-25T20:57:59.941 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:00.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:00 smithi037 ceph-mon[29195]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:01.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 -- 172.21.15.37:0/1843806151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 msgr2=0x7f3ec4103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/1843806151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4103a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009910 tx=0x7f3eb4009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 -- 172.21.15.37:0/1843806151 shutdown_connections 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/1843806151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4103a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 -- 172.21.15.37:0/1843806151 >> 172.21.15.37:0/1843806151 conn(0x7f3ec40fcea0 msgr2=0x7f3ec40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 -- 172.21.15.37:0/1843806151 shutdown_connections 2024-04-25T20:58:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.446+0000 7f3ec8af2700 1 -- 172.21.15.37:0/1843806151 wait complete. 2024-04-25T20:58:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.447+0000 7f3ec8af2700 1 Processor -- start 2024-04-25T20:58:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.447+0000 7f3ec8af2700 1 -- start start 2024-04-25T20:58:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.447+0000 7f3ec8af2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.447+0000 7f3ec8af2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec410c9d0 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57870/0 (socket says 172.21.15.37:57870) 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec259c700 1 -- 172.21.15.37:0/2701335050 learned_addr learned my addr 172.21.15.37:0/2701335050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec259c700 1 -- 172.21.15.37:0/2701335050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec41984b0 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec259c700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40056d0 tx=0x7f3eb40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4026070 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb4004370 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec4198740 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb40039b0 con 0x7f3ec4103650 2024-04-25T20:58:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.448+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec4198ba0 con 0x7f3ec4103650 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.449+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3eb4029070 con 0x7f3ec4103650 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.449+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec406b190 con 0x7f3ec4103650 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.449+0000 7f3ebb7fe700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 0x7f3eb003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.449+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3eb401e070 con 0x7f3ec4103650 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.450+0000 7f3ebbfff700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 0x7f3eb003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:01.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.450+0000 7f3ebbfff700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 0x7f3eb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.455+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3eb4013070 con 0x7f3ec4103650 2024-04-25T20:58:01.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:01.671+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}) -- 0x7f3ec4199340 con 0x7f3eb00396b0 2024-04-25T20:58:01.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:01 smithi037 ceph-mon[29195]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:02.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:02 smithi037 ceph-mon[29195]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:03.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:03.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: Deploying cephadm binary to smithi177 2024-04-25T20:58:03.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:03.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:03 smithi037 ceph-mon[29195]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.976+0000 7f3ebb7fe700 1 -- 172.21.15.37:0/2701335050 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f3ec4199340 con 0x7f3eb00396b0 2024-04-25T20:58:05.977 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi177' with addr '172.21.15.177' 2024-04-25T20:58:05.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:05 smithi037 ceph-mon[29195]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 msgr2=0x7f3eb003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 0x7f3eb003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3eac006fd0 tx=0x7f3eac006e40 comp rx=0 tx=0).stop 2024-04-25T20:58:05.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 msgr2=0x7f3ec4197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40056d0 tx=0x7f3eb40057b0 comp rx=0 tx=0).stop 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 shutdown_connections 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3eb00396b0 0x7f3eb003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 --2- 172.21.15.37:0/2701335050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ec4103650 0x7f3ec4197f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 >> 172.21.15.37:0/2701335050 conn(0x7f3ec40fcea0 msgr2=0x7f3ec41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 shutdown_connections 2024-04-25T20:58:05.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:05.978+0000 7f3ec8af2700 1 -- 172.21.15.37:0/2701335050 wait complete. 2024-04-25T20:58:06.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:58:06.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:06.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:06.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:06.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:06.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:06.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:06.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: Added host smithi177 2024-04-25T20:58:06.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:06.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:07.001 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:08.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:07 smithi037 ceph-mon[29195]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:08.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:07 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/2712475002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 msgr2=0x7fa3dc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/2712475002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc1089f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009910 tx=0x7fa3cc009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/2712475002 shutdown_connections 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/2712475002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc1089f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/2712475002 >> 172.21.15.37:0/2712475002 conn(0x7fa3dc078970 msgr2=0x7fa3dc078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/2712475002 shutdown_connections 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.488+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/2712475002 wait complete. 2024-04-25T20:58:08.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.489+0000 7fa3e0ddf700 1 Processor -- start 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.489+0000 7fa3e0ddf700 1 -- start start 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.489+0000 7fa3e0ddf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.489+0000 7fa3e0ddf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3dc10cac0 con 0x7fa3dc108610 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57890/0 (socket says 172.21.15.37:57890) 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3da59c700 1 -- 172.21.15.37:0/1083767506 learned_addr learned my addr 172.21.15.37:0/1083767506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3da59c700 1 -- 172.21.15.37:0/1083767506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3dc1983d0 con 0x7fa3dc108610 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3da59c700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0056d0 tx=0x7fa3cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3cc026070 con 0x7fa3dc108610 2024-04-25T20:58:08.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3cc00bc00 con 0x7fa3dc108610 2024-04-25T20:58:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3dc198660 con 0x7fa3dc108610 2024-04-25T20:58:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3cc003920 con 0x7fa3dc108610 2024-04-25T20:58:08.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.490+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3dc198ac0 con 0x7fa3dc108610 2024-04-25T20:58:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.491+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa3cc029070 con 0x7fa3dc108610 2024-04-25T20:58:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.491+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3dc06a630 con 0x7fa3dc108610 2024-04-25T20:58:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.491+0000 7fa3d3fff700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 0x7fa3c803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.491+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa3cc01e070 con 0x7fa3dc108610 2024-04-25T20:58:08.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.492+0000 7fa3d1d9b700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 0x7fa3c803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:08.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.492+0000 7fa3d1d9b700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 0x7fa3c803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:08.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.497+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3cc013070 con 0x7fa3dc108610 2024-04-25T20:58:08.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.723+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3dc199260 con 0x7fa3c80396b0 2024-04-25T20:58:08.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.724+0000 7fa3d3fff700 1 -- 172.21.15.37:0/1083767506 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fa3dc199260 con 0x7fa3c80396b0 2024-04-25T20:58:08.726 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T20:58:08.726 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.98", "hostname": "smithi098", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}] 2024-04-25T20:58:08.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.726+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 msgr2=0x7fa3c803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.726+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 0x7fa3c803bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4006fd0 tx=0x7fa3c4006e40 comp rx=0 tx=0).stop 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.726+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 msgr2=0x7fa3dc197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.726+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0056d0 tx=0x7fa3cc0057b0 comp rx=0 tx=0).stop 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 shutdown_connections 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa3c80396b0 0x7fa3c803bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 --2- 172.21.15.37:0/1083767506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3dc108610 0x7fa3dc197e90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 >> 172.21.15.37:0/1083767506 conn(0x7fa3dc078970 msgr2=0x7fa3dc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 shutdown_connections 2024-04-25T20:58:08.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:08.727+0000 7fa3e0ddf700 1 -- 172.21.15.37:0/1083767506 wait complete. 2024-04-25T20:58:09.292 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T20:58:09.292 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T20:58:09.532 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:09.823 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:09 smithi037 ceph-mon[29195]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:09.823 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:09.823 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:09 smithi037 ceph-mon[29195]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/3791074993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 msgr2=0x7f0ec4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/3791074993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4101990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009910 tx=0x7f0eb4009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:10.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/3791074993 shutdown_connections 2024-04-25T20:58:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/3791074993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4101990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/3791074993 >> 172.21.15.37:0/3791074993 conn(0x7f0ec40fce40 msgr2=0x7f0ec40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/3791074993 shutdown_connections 2024-04-25T20:58:10.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.985+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/3791074993 wait complete. 2024-04-25T20:58:10.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec8dc4700 1 Processor -- start 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec8dc4700 1 -- start start 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec8dc4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec8dc4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec410b830 con 0x7f0ec41015b0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42408/0 (socket says 172.21.15.37:42408) 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.986+0000 7f0ec259c700 1 -- 172.21.15.37:0/466473198 learned_addr learned my addr 172.21.15.37:0/466473198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0ec259c700 1 -- 172.21.15.37:0/466473198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec41983e0 con 0x7f0ec41015b0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0ec259c700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40056d0 tx=0x7f0eb40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4026070 con 0x7f0ec41015b0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb400bba0 con 0x7f0ec41015b0 2024-04-25T20:58:10.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec41986d0 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0eb4003920 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.987+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec419c770 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0eb4029070 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec4102a90 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0eb37fe700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 0x7f0eac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0eb401e070 con 0x7f0ec41015b0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0ec1d9b700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 0x7f0eac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.988+0000 7f0ec1d9b700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 0x7f0eac03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8006fd0 tx=0x7f0eb8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:10.993+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0eb4013070 con 0x7f0ec41015b0 2024-04-25T20:58:11.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.205+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f0ec40640a0 con 0x7f0ec41015b0 2024-04-25T20:58:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.549+0000 7f0eb37fe700 1 -- 172.21.15.37:0/466473198 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f0eb4030090 con 0x7f0ec41015b0 2024-04-25T20:58:11.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 msgr2=0x7f0eac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 0x7f0eac03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8006fd0 tx=0x7f0eb8006e40 comp rx=0 tx=0).stop 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 msgr2=0x7f0ec4197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40056d0 tx=0x7f0eb40057b0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 shutdown_connections 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0eac039650 0x7f0eac03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 --2- 172.21.15.37:0/466473198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ec41015b0 0x7f0ec4197ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.551+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 >> 172.21.15.37:0/466473198 conn(0x7f0ec40fce40 msgr2=0x7f0ec406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.552+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 shutdown_connections 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:11.552+0000 7f0ec8dc4700 1 -- 172.21.15.37:0/466473198 wait complete. 2024-04-25T20:58:11.553 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-25T20:58:11.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:11 smithi037 ceph-mon[29195]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:11.840 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:11 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:58:12.138 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-04-25T20:58:12.138 INFO:tasks.cephadm:Adding mon.b on smithi098 2024-04-25T20:58:12.138 INFO:tasks.cephadm:Adding mon.c on smithi177 2024-04-25T20:58:12.138 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c' 2024-04-25T20:58:12.371 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T20:58:12.419 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:12.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:12.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:12 smithi037 ceph-mon[29195]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:13.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 -- 172.21.15.177:0/3303667066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 msgr2=0x7fe844101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:13.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/3303667066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844101970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe84000b0a0 tx=0x7fe84000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 -- 172.21.15.177:0/3303667066 shutdown_connections 2024-04-25T20:58:13.945 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/3303667066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844101970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 -- 172.21.15.177:0/3303667066 >> 172.21.15.177:0/3303667066 conn(0x7fe8440fce40 msgr2=0x7fe8440ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 -- 172.21.15.177:0/3303667066 shutdown_connections 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.943+0000 7fe84b3a7700 1 -- 172.21.15.177:0/3303667066 wait complete. 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe84b3a7700 1 Processor -- start 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe84b3a7700 1 -- start start 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe84b3a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe84b3a7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe84410b810 con 0x7fe844101590 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe849143700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe849143700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.177:45158/0 (socket says 172.21.15.177:45158) 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.944+0000 7fe849143700 1 -- 172.21.15.177:0/1065088051 learned_addr learned my addr 172.21.15.177:0/1065088051 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe849143700 1 -- 172.21.15.177:0/1065088051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe844198350 con 0x7fe844101590 2024-04-25T20:58:13.946 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe849143700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe840006130 tx=0x7fe8400097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe840021020 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8441985e0 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8400079b0 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84419c6e0 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.945+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe840017040 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.946+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe840024020 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.946+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe844102a70 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.946+0000 7fe83a7fc700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 0x7fe83003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.946+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe84000f050 con 0x7fe844101590 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.947+0000 7fe848942700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 0x7fe83003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:13.948 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.947+0000 7fe848942700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 0x7fe83003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe834006fd0 tx=0x7fe834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:13.952 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:13.952+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe84002b020 con 0x7fe844101590 2024-04-25T20:58:14.163 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.162+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}) -- 0x7fe8440640a0 con 0x7fe830039650 2024-04-25T20:58:14.172 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.171+0000 7fe83a7fc700 1 -- 172.21.15.177:0/1065088051 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe8440640a0 con 0x7fe830039650 2024-04-25T20:58:14.172 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mon update... 2024-04-25T20:58:14.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 msgr2=0x7fe83003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:14.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 0x7fe83003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe834006fd0 tx=0x7fe834006e40 comp rx=0 tx=0).stop 2024-04-25T20:58:14.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 msgr2=0x7fe844197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:14.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe840006130 tx=0x7fe8400097d0 comp rx=0 tx=0).stop 2024-04-25T20:58:14.174 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 shutdown_connections 2024-04-25T20:58:14.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe830039650 0x7fe83003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:14.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 --2- 172.21.15.177:0/1065088051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe844101590 0x7fe844197e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:14.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 >> 172.21.15.177:0/1065088051 conn(0x7fe8440fce40 msgr2=0x7fe84406a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:14.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 shutdown_connections 2024-04-25T20:58:14.175 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:14.173+0000 7fe84b3a7700 1 -- 172.21.15.177:0/1065088051 wait complete. 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:14.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:14.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:14.759 DEBUG:teuthology.orchestra.run.smithi098:mon.b> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T20:58:14.762 DEBUG:teuthology.orchestra.run.smithi177:mon.c> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T20:58:14.764 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:58:14.765 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:58:15.475 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:15.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:15.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c;count:3 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:15.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:15 smithi037 ceph-mon[29195]: Deploying daemon mon.c on smithi177 2024-04-25T20:58:17.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:17 smithi037 ceph-mon[29195]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 -- 172.21.15.177:0/3552324966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 msgr2=0x7fce201017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 --2- 172.21.15.177:0/3552324966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce201017e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce10009910 tx=0x7fce10009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 -- 172.21.15.177:0/3552324966 shutdown_connections 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 --2- 172.21.15.177:0/3552324966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce201017e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 -- 172.21.15.177:0/3552324966 >> 172.21.15.177:0/3552324966 conn(0x7fce200fce40 msgr2=0x7fce200ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:18.295 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 -- 172.21.15.177:0/3552324966 shutdown_connections 2024-04-25T20:58:18.296 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.293+0000 7fce24b83700 1 -- 172.21.15.177:0/3552324966 wait complete. 2024-04-25T20:58:18.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.294+0000 7fce24b83700 1 Processor -- start 2024-04-25T20:58:18.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.294+0000 7fce24b83700 1 -- start start 2024-04-25T20:58:18.297 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.294+0000 7fce24b83700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:18.298 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.294+0000 7fce24b83700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce20112fb0 con 0x7fce20101400 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.294+0000 7fce1e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce1e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.177:45162/0 (socket says 172.21.15.177:45162) 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce1e59c700 1 -- 172.21.15.177:0/1927186077 learned_addr learned my addr 172.21.15.177:0/1927186077 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce1e59c700 1 -- 172.21.15.177:0/1927186077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2019ea60 con 0x7fce20101400 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce1e59c700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce100056d0 tx=0x7fce100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:18.300 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce10026070 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2019ecf0 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce10004370 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce100039b0 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.295+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2019f150 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.296+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2006b190 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.296+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fce10029070 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.297+0000 7fce0f7fe700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 0x7fce0803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.297+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fce1001e070 con 0x7fce20101400 2024-04-25T20:58:18.301 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.297+0000 7fce1dd9b700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 0x7fce0803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:18.302 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.297+0000 7fce1dd9b700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 0x7fce0803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fce14006fd0 tx=0x7fce14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:18.303 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.302+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce10013070 con 0x7fce20101400 2024-04-25T20:58:18.608 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.608+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fce2019f880 con 0x7fce20101400 2024-04-25T20:58:18.609 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.608+0000 7fce0f7fe700 1 -- 172.21.15.177:0/1927186077 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fce10030090 con 0x7fce20101400 2024-04-25T20:58:18.609 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:58:18.609 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":1,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","modified":"2024-04-25T20:56:26.923204Z","created":"2024-04-25T20:56:26.923204Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T20:58:18.611 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 msgr2=0x7fce0803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 0x7fce0803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fce14006fd0 tx=0x7fce14006e40 comp rx=0 tx=0).stop 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 msgr2=0x7fce2019e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce100056d0 tx=0x7fce100057b0 comp rx=0 tx=0).stop 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 shutdown_connections 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fce08039650 0x7fce0803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 --2- 172.21.15.177:0/1927186077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce20101400 0x7fce2019e520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 >> 172.21.15.177:0/1927186077 conn(0x7fce200fce40 msgr2=0x7fce200fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 shutdown_connections 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:18.611+0000 7fce24b83700 1 -- 172.21.15.177:0/1927186077 wait complete. 2024-04-25T20:58:18.612 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 1 2024-04-25T20:58:18.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:18 smithi177 systemd[1]: Starting Ceph mon.c for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T20:58:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:19 smithi037 ceph-mon[29195]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:19 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/1927186077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:19.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 podman[33025]: 2024-04-25 20:58:19.513580348 +0000 UTC m=+0.017989102 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:58:19.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 podman[33025]: 2024-04-25 20:58:19.6182505 +0000 UTC m=+0.122659291 container create 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T20:58:19.880 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 podman[33025]: 2024-04-25 20:58:19.829325948 +0000 UTC m=+0.333734706 container init 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:58:19.880 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 podman[33025]: 2024-04-25 20:58:19.832565224 +0000 UTC m=+0.336973973 container start 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pidfile_write: ignore empty --pid-file 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: load: jerasure load: lrc 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Git sha 0 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: DB SUMMARY 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: DB Session ID: 6T0ITQPNI7E9ZJW3FC5A 2024-04-25T20:58:19.881 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.env: 0x557a120ffcc0 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:58:19.882 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.info_log: 0x557a14631be0 2024-04-25T20:58:19.883 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.statistics: (nil) 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.use_fsync: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:58:19.884 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.db_log_dir: 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.wal_dir: 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.write_buffer_manager: 0x557a145c50e0 2024-04-25T20:58:19.885 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.unordered_write: 0 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:58:19.886 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.row_cache: None 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.wal_filter: None 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.wal_compression: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:58:19.887 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:58:19.888 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_open_files: -1 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Compression algorithms supported: 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kZSTD supported: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:58:19.889 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:58:19.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.merge_operator: 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_filter: None 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557a14631dc0) 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:58:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_type: 0 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_index_type: 0 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_shortening: 1 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: checksum: 4 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: no_block_cache: 0 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache: 0x557a145f71f0 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_options: 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: capacity : 536870912 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_shard_bits : 4 2024-04-25T20:58:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout: strict_capacity_limit : 0 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_compressed: (nil) 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: persistent_cache: (nil) 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size: 4096 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size_deviation: 10 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_restart_interval: 16 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_block_restart_interval: 1 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: metadata_block_size: 4096 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: partition_filters: 0 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: use_delta_encoding: 1 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: filter_policy: bloomfilter 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: whole_key_filtering: 1 2024-04-25T20:58:19.893 INFO:journalctl@ceph.mon.c.smithi177.stdout: verify_compression: 0 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: format_version: 5 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_index_compression: 1 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_align: 0 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: max_auto_readahead_size: 262144 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: prepopulate_block_cache: 0 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression: NoCompression 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.num_levels: 7 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:58:19.895 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:58:19.896 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:58:19.897 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:58:19.898 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:58:19.899 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:58:19.900 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:58:19.901 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:58:19.902 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.ttl: 2592000 2024-04-25T20:58:19.902 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:58:19.902 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:58:19.903 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:58:19.903 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:58:19.903 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:58:19.903 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:58:19.903 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T20:58:19.904 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e420a927-c243-49f4-b77d-b13e2a558457 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078699862308, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078699862719, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078699, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e420a927-c243-49f4-b77d-b13e2a558457", "db_session_id": "6T0ITQPNI7E9ZJW3FC5A", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078699862798, "job": 1, "event": "recovery_finished"} 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557a1470c000 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: DB pointer 0x557a146f6000 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** DB Stats ** 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:19.905 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:19.906 INFO:journalctl@ceph.mon.c.smithi177.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:58:19.907 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:58:19.908 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache BinnedLRUCache@0x557a145f71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T20:58:19.909 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: using public_addr v2:172.21.15.177:0/0 -> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: starting mon.c rank -1 at public addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] at bind addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(???) e0 preinit fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout: e1 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:58:19.910 INFO:journalctl@ceph.mon.c.smithi177.stdout: legacy client fscid: -1 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout: No filesystems configured 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T20:58:19.911 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.912 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Added host smithi098 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:19.913 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Deploying cephadm binary to smithi177 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:19.914 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Added host smithi177 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.915 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:19.916 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c;count:3 2024-04-25T20:58:19.917 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: Deploying daemon mon.c on smithi177 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/1927186077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:19.918 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 ceph-mon[33043]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T20:58:20.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 bash[33025]: 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a 2024-04-25T20:58:20.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:19 smithi177 systemd[1]: Started Ceph mon.c for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:58:20.395 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:58:20.395 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:58:20.629 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:58:25.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 -- 172.21.15.177:0/2065434061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 msgr2=0x7ff868005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:25.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 --2- 172.21.15.177:0/2065434061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff868005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009bf0 tx=0x7ff86c009f00 comp rx=0 tx=0).stop 2024-04-25T20:58:25.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 -- 172.21.15.177:0/2065434061 shutdown_connections 2024-04-25T20:58:25.110 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 --2- 172.21.15.177:0/2065434061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff868005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 -- 172.21.15.177:0/2065434061 >> 172.21.15.177:0/2065434061 conn(0x7ff87c0778c0 msgr2=0x7ff87c077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:25.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 -- 172.21.15.177:0/2065434061 shutdown_connections 2024-04-25T20:58:25.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.109+0000 7ff8815e4700 1 -- 172.21.15.177:0/2065434061 wait complete. 2024-04-25T20:58:25.111 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 Processor -- start 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 -- start start 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff87c196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1093c0 con 0x7ff87c106420 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8815e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c109240 con 0x7ff87c197080 2024-04-25T20:58:25.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:35914/0 (socket says 172.21.15.177:35914) 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.110+0000 7ff8727fc700 1 -- 172.21.15.177:0/3866374841 learned_addr learned my addr 172.21.15.177:0/3866374841 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff87affd700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff87c196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8727fc700 1 -- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 msgr2=0x7ff87c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8727fc700 1 -- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 msgr2=0x7ff87c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8727fc700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8727fc700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff87affd700 1 -- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 msgr2=0x7ff87c197500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff87affd700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff87affd700 1 -- 172.21.15.177:0/3866374841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c101a70 con 0x7ff87c106420 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff87affd700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff87c196b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009bf0 tx=0x7ff86c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff86c004a30 con 0x7ff87c106420 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86c024030 con 0x7ff87c106420 2024-04-25T20:58:25.113 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1a41b0 con 0x7ff87c106420 2024-04-25T20:58:25.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff86c017470 con 0x7ff87c106420 2024-04-25T20:58:25.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.111+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1a4790 con 0x7ff87c106420 2024-04-25T20:58:25.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.112+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff86c01e070 con 0x7ff87c106420 2024-04-25T20:58:25.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.112+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c102d30 con 0x7ff87c106420 2024-04-25T20:58:25.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.113+0000 7ff878ff9700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 0x7ff85c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:25.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.113+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff86c01c070 con 0x7ff87c106420 2024-04-25T20:58:25.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.113+0000 7ff8727fc700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 0x7ff85c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:25.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.113+0000 7ff8727fc700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 0x7ff85c03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff864009180 tx=0x7ff864008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:25.118 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.118+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff86c025280 con 0x7ff87c106420 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: monmap epoch 2 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: last_changed 2024-04-25T20:58:19.887457+0000 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:25.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: fsmap 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: mgrmap e13: x(active, since 50s) 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:25.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: mon.a calling monitor election 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: mon.c calling monitor election 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: monmap epoch 2 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: last_changed 2024-04-25T20:58:19.887457+0000 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: min_mon_release 19 (squid) 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: election_strategy: 1 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: fsmap 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: mgrmap e13: x(active, since 50s) 2024-04-25T20:58:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: overall HEALTH_OK 2024-04-25T20:58:25.343 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.343 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.343 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:25.343 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:25.343 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:25.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.397+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff87c02df80 con 0x7ff87c106420 2024-04-25T20:58:25.398 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.398+0000 7ff878ff9700 1 -- 172.21.15.177:0/3866374841 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff86c01fde0 con 0x7ff87c106420 2024-04-25T20:58:25.399 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:58:25.399 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":2,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","modified":"2024-04-25T20:58:19.887457Z","created":"2024-04-25T20:56:26.923204Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 msgr2=0x7ff85c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 0x7ff85c03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff864009180 tx=0x7ff864008040 comp rx=0 tx=0).stop 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 msgr2=0x7ff87c196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff87c196b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009bf0 tx=0x7ff86c0044b0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 shutdown_connections 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff85c0393f0 0x7ff85c03b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff87c106420 0x7ff87c196b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 --2- 172.21.15.177:0/3866374841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff87c197080 0x7ff87c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 >> 172.21.15.177:0/3866374841 conn(0x7ff87c0778c0 msgr2=0x7ff87c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 shutdown_connections 2024-04-25T20:58:25.401 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:25.400+0000 7ff8815e4700 1 -- 172.21.15.177:0/3866374841 wait complete. 2024-04-25T20:58:25.402 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 2 2024-04-25T20:58:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:25 smithi037 ceph-mon[29195]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:25 smithi037 ceph-mon[29195]: Deploying daemon mon.b on smithi098 2024-04-25T20:58:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:25 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/3866374841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:26.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:25 smithi177 ceph-mon[33043]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:25 smithi177 ceph-mon[33043]: Deploying daemon mon.b on smithi098 2024-04-25T20:58:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:25 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/3866374841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:26.958 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:58:26.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:58:27.198 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:58:28.228 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:27 smithi177 ceph-mon[33043]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:28.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:27 smithi037 ceph-mon[29195]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:28.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 podman[32543]: 2024-04-25 20:58:28.627738639 +0000 UTC m=+0.147522940 container create ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:58:28.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- 172.21.15.177:0/961707039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0108630 msgr2=0x7ff1b0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:28.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/961707039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0108630 0x7ff1b0108a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009910 tx=0x7ff1a0009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:28.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- 172.21.15.177:0/961707039 shutdown_connections 2024-04-25T20:58:28.741 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/961707039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0108630 0x7ff1b0108a10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- 172.21.15.177:0/961707039 >> 172.21.15.177:0/961707039 conn(0x7ff1b00fe050 msgr2=0x7ff1b0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- 172.21.15.177:0/961707039 shutdown_connections 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- 172.21.15.177:0/961707039 wait complete. 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 Processor -- start 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.739+0000 7ff1b7de2700 1 -- start start 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b7de2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b7de2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 0x7ff1b019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b7de2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b010caa0 con 0x7ff1b0198660 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b7de2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b010c920 con 0x7ff1b0108630 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:35926/0 (socket says 172.21.15.177:35926) 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 -- 172.21.15.177:0/3231475769 learned_addr learned my addr 172.21.15.177:0/3231475769 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:58:28.742 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b537d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 0x7ff1b019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 -- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 msgr2=0x7ff1b0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 -- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 msgr2=0x7ff1b0198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b5b7e700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b537d700 1 -- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 msgr2=0x7ff1b0198120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b537d700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b537d700 1 -- 172.21.15.177:0/3231475769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b0198ca0 con 0x7ff1b0198660 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1b537d700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 0x7ff1b019c9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00b4d0 tx=0x7ff1ac00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.740+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff1ac011630 con 0x7ff1b0198660 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b019cf30 con 0x7ff1b0198660 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1ac011c80 con 0x7ff1b0198660 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff1ac010da0 con 0x7ff1b0198660 2024-04-25T20:58:28.743 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b019d510 con 0x7ff1b0198660 2024-04-25T20:58:28.744 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b019d820 con 0x7ff1b0198660 2024-04-25T20:58:28.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.741+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff1ac018020 con 0x7ff1b0198660 2024-04-25T20:58:28.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.742+0000 7ff1a6ffd700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 0x7ff19c03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:28.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.742+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff1ac04c2b0 con 0x7ff1b0198660 2024-04-25T20:58:28.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.742+0000 7ff1b5b7e700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 0x7ff19c03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:28.745 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.742+0000 7ff1b5b7e700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 0x7ff19c03b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009750 tx=0x7ff1a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:28.748 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.747+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1b019d820 con 0x7ff1b0198660 2024-04-25T20:58:28.864 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:28.864+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac017bc0 con 0x7ff1b0198660 2024-04-25T20:58:29.030 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:29.029+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff1b0199480 con 0x7ff1b0198660 2024-04-25T20:58:29.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 podman[32543]: 2024-04-25 20:58:28.806532651 +0000 UTC m=+0.326316954 container init ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-04-25T20:58:29.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 podman[32543]: 2024-04-25 20:58:28.80956401 +0000 UTC m=+0.329348314 container start ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T20:58:29.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pidfile_write: ignore empty --pid-file 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: load: jerasure load: lrc 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Git sha 0 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: DB SUMMARY 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: DB Session ID: GQCZUYP2J7O26V49QJ2B 2024-04-25T20:58:29.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.env: 0x560604128cc0 2024-04-25T20:58:29.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:58:29.171 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.info_log: 0x560606821be0 2024-04-25T20:58:29.171 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:58:29.172 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.statistics: (nil) 2024-04-25T20:58:29.172 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.use_fsync: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:58:29.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.db_log_dir: 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.wal_dir: 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:58:29.174 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.write_buffer_manager: 0x5606067b50e0 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:58:29.175 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.unordered_write: 0 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.row_cache: None 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.wal_filter: None 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:58:29.176 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.wal_compression: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:58:29.177 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:58:29.178 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_open_files: -1 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Compression algorithms supported: 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kZSTD supported: 0 2024-04-25T20:58:29.179 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.merge_operator: 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_filter: None 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:58:29.180 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560606821dc0) 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_type: 0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_index_type: 0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_shortening: 1 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: checksum: 4 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: no_block_cache: 0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache: 0x5606067e71f0 2024-04-25T20:58:29.181 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_options: 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: capacity : 536870912 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_shard_bits : 4 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: strict_capacity_limit : 0 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_cache_compressed: (nil) 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: persistent_cache: (nil) 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size: 4096 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_size_deviation: 10 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_restart_interval: 16 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: index_block_restart_interval: 1 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: metadata_block_size: 4096 2024-04-25T20:58:29.182 INFO:journalctl@ceph.mon.b.smithi098.stdout: partition_filters: 0 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: use_delta_encoding: 1 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: filter_policy: bloomfilter 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: whole_key_filtering: 1 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: verify_compression: 0 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: format_version: 5 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_index_compression: 1 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: block_align: 0 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: max_auto_readahead_size: 262144 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: prepopulate_block_cache: 0 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:58:29.183 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression: NoCompression 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.num_levels: 7 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:58:29.184 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:58:29.185 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:58:29.186 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:58:29.187 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:58:29.188 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.ttl: 2592000 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:58:29.189 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 57ab06f1-df33-49c4-a591-bc38f6d82318 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078708839257, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078708839675, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078708, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "57ab06f1-df33-49c4-a591-bc38f6d82318", "db_session_id": "GQCZUYP2J7O26V49QJ2B", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078708839754, "job": 1, "event": "recovery_finished"} 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5606068fc000 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: DB pointer 0x5606068e6000 2024-04-25T20:58:29.190 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** DB Stats ** 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:58:29.191 INFO:journalctl@ceph.mon.b.smithi098.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** Compaction Stats [default] ** 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:58:29.192 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache BinnedLRUCache@0x5606067e71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: using public_addr v2:172.21.15.98:0/0 -> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: starting mon.b rank -1 at public addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] at bind addrs [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:29.193 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(???) e0 preinit fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: e1 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: legacy client fscid: -1 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout: No filesystems configured 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T20:58:29.194 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Added host smithi098 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.195 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Deploying cephadm binary to smithi177 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi098", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:29.196 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Added host smithi177 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.197 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/466473198' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.198 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.199 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c;count:3 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Deploying daemon mon.c on smithi177 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/1927186077' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:29.200 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.a calling monitor election 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.c calling monitor election 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: monmap epoch 2 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: last_changed 2024-04-25T20:58:19.887457+0000 2024-04-25T20:58:29.201 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: min_mon_release 19 (squid) 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: election_strategy: 1 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: fsmap 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mgrmap e13: x(active, since 50s) 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:29.202 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: Deploying daemon mon.b on smithi098 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/3866374841' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 ceph-mon[32569]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 bash[32543]: ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 2024-04-25T20:58:29.203 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:28 smithi098 systemd[1]: Started Ceph mon.b for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:58:33.875 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.873+0000 7ff1a6ffd700 1 -- 172.21.15.177:0/3231475769 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7ff1ac01f020 con 0x7ff1b0198660 2024-04-25T20:58:33.875 INFO:teuthology.orchestra.run.smithi177.stdout: 2024-04-25T20:58:33.875 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":3,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","modified":"2024-04-25T20:58:28.863619Z","created":"2024-04-25T20:56:26.923204Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:3300","nonce":0},{"type":"v1","addr":"172.21.15.98:6789","nonce":0}]},"addr":"172.21.15.98:6789/0","public_addr":"172.21.15.98:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 msgr2=0x7ff19c03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 0x7ff19c03b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009750 tx=0x7ff1a001a040 comp rx=0 tx=0).stop 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 msgr2=0x7ff1b019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 0x7ff1b019c9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00b4d0 tx=0x7ff1ac00b890 comp rx=0 tx=0).stop 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 shutdown_connections 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff19c0393f0 0x7ff19c03b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b0108630 0x7ff1b0198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 --2- 172.21.15.177:0/3231475769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1b0198660 0x7ff1b019c9f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 >> 172.21.15.177:0/3231475769 conn(0x7ff1b00fe050 msgr2=0x7ff1b00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.879 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 shutdown_connections 2024-04-25T20:58:33.880 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:33.877+0000 7ff1b7de2700 1 -- 172.21.15.177:0/3231475769 wait complete. 2024-04-25T20:58:33.880 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 3 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: mon.a calling monitor election 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: mon.c calling monitor election 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/3231475769' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.209 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: mon.b calling monitor election 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: monmap epoch 3 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: last_changed 2024-04-25T20:58:28.863619+0000 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: min_mon_release 19 (squid) 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: election_strategy: 1 2024-04-25T20:58:34.210 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: fsmap 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: mgrmap e13: x(active, since 59s) 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: overall HEALTH_OK 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.211 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: mon.a calling monitor election 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: mon.c calling monitor election 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/3231475769' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: mon.b calling monitor election 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: monmap epoch 3 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: last_changed 2024-04-25T20:58:28.863619+0000 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: min_mon_release 19 (squid) 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: election_strategy: 1 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: fsmap 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:34.237 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: mgrmap e13: x(active, since 59s) 2024-04-25T20:58:34.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T20:58:34.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:58:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: mon.a calling monitor election 2024-04-25T20:58:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: mon.c calling monitor election 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/3231475769' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: mon.b calling monitor election 2024-04-25T20:58:34.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: monmap epoch 3 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: last_changed 2024-04-25T20:58:28.863619+0000 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: created 2024-04-25T20:56:26.923204+0000 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: min_mon_release 19 (squid) 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: election_strategy: 1 2024-04-25T20:58:34.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: 1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.c 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: 2: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.b 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: fsmap 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: mgrmap e13: x(active, since 59s) 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:34.545 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T20:58:34.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T20:58:34.802 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:35.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:35.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.194 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:35.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:35.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:35.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:58:36.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:36.199 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:36.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:36.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:36.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 -- 172.21.15.37:0/1644528633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 msgr2=0x7f5674101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 --2- 172.21.15.37:0/1644528633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674101970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5664009910 tx=0x7f5664009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 -- 172.21.15.37:0/1644528633 shutdown_connections 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 --2- 172.21.15.37:0/1644528633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674101970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 -- 172.21.15.37:0/1644528633 >> 172.21.15.37:0/1644528633 conn(0x7f56740fce40 msgr2=0x7f56740ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 -- 172.21.15.37:0/1644528633 shutdown_connections 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.523+0000 7f5679d84700 1 -- 172.21.15.37:0/1644528633 wait complete. 2024-04-25T20:58:36.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 Processor -- start 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 -- start start 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5674103570 0x7f56741039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 0x7f56741a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f567410b810 con 0x7f5674101590 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f567410b690 con 0x7f56741a43a0 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5679d84700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f567410b990 con 0x7f5674103570 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f56737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f56737fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59344/0 (socket says 172.21.15.37:59344) 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f56737fe700 1 -- 172.21.15.37:0/564450443 learned_addr learned my addr 172.21.15.37:0/564450443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.524+0000 7f5672ffd700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5674103570 0x7f56741039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5673fff700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 0x7f56741a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 -- 172.21.15.37:0/564450443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5674103570 msgr2=0x7f56741039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5674103570 0x7f56741039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 -- 172.21.15.37:0/564450443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 msgr2=0x7f56741a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 0x7f56741a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 -- 172.21.15.37:0/564450443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a6e60 con 0x7f5674101590 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5673fff700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 0x7f56741a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f56737fe700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5664004f80 tx=0x7f5664004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664026070 con 0x7f5674101590 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566400b6f0 con 0x7f5674101590 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a7150 con 0x7f5674101590 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640038c0 con 0x7f5674101590 2024-04-25T20:58:36.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.525+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a76e0 con 0x7f5674101590 2024-04-25T20:58:36.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.526+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5664029070 con 0x7f5674101590 2024-04-25T20:58:36.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.526+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5674102a70 con 0x7f5674101590 2024-04-25T20:58:36.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.526+0000 7f5670ff9700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 0x7f565c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:36.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.527+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f566404d320 con 0x7f5674101590 2024-04-25T20:58:36.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.527+0000 7f5672ffd700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 0x7f565c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:36.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.527+0000 7f5672ffd700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 0x7f565c03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5668009ad0 tx=0x7f5668009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:36.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.532+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5664013070 con 0x7f5674101590 2024-04-25T20:58:36.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.748+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f56740640a0 con 0x7f5674101590 2024-04-25T20:58:36.750 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:36.750 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T20:58:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T20:58:36.750 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2024-04-25T20:58:36.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.748+0000 7f5670ff9700 1 -- 172.21.15.37:0/564450443 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f5664030090 con 0x7f5674101590 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 msgr2=0x7f565c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 0x7f565c03bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5668009ad0 tx=0x7f5668009210 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 msgr2=0x7f5674103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5664004f80 tx=0x7f5664004fb0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 shutdown_connections 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f565c039ae0 0x7f565c03bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56741a43a0 0x7f56741a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5674101590 0x7f5674103030 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 --2- 172.21.15.37:0/564450443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5674103570 0x7f56741039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 >> 172.21.15.37:0/564450443 conn(0x7f56740fce40 msgr2=0x7f567406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 shutdown_connections 2024-04-25T20:58:36.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:36.751+0000 7f5679d84700 1 -- 172.21.15.37:0/564450443 wait complete. 2024-04-25T20:58:36.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T20:58:36.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:36.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:36 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/564450443' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:37.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:36 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/564450443' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:37.418 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T20:58:37.419 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:58:37.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:58:37.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T20:58:37.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.420 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:37.421 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:37.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:37.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:37.422 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:36 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/564450443' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:37.449 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:58:37.449 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:37.514 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:58:37.515 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:58:37.546 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:58:37.547 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:37.615 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:58:37.615 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:58:37.646 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:58:37.646 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:58:37.713 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-04-25T20:58:37.714 INFO:tasks.cephadm:Adding mgr.y on smithi098 2024-04-25T20:58:37.714 INFO:tasks.cephadm:Adding mgr.z on smithi177 2024-04-25T20:58:37.714 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi098=y;smithi177=z' 2024-04-25T20:58:37.986 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:58:38.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:37 smithi037 ceph-mon[29195]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:38.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:37 smithi037 ceph-mon[29195]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:58:38.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:37 smithi037 ceph-mon[29195]: Reconfiguring daemon mon.a on smithi037 2024-04-25T20:58:38.290 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:37 smithi177 ceph-mon[33043]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:38.290 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:37 smithi177 ceph-mon[33043]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:58:38.290 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:37 smithi177 ceph-mon[33043]: Reconfiguring daemon mon.a on smithi037 2024-04-25T20:58:38.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:37 smithi098 ceph-mon[32569]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:38.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:37 smithi098 ceph-mon[32569]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:58:38.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:37 smithi098 ceph-mon[32569]: Reconfiguring daemon mon.a on smithi037 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 -- 172.21.15.177:0/2469563500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401025d0 msgr2=0x7f24401029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 --2- 172.21.15.177:0/2469563500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401025d0 0x7f24401029f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2434009910 tx=0x7f2434009c20 comp rx=0 tx=0).stop 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 -- 172.21.15.177:0/2469563500 shutdown_connections 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 --2- 172.21.15.177:0/2469563500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401025d0 0x7f24401029f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 -- 172.21.15.177:0/2469563500 >> 172.21.15.177:0/2469563500 conn(0x7f24400fdb70 msgr2=0x7f24400fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 -- 172.21.15.177:0/2469563500 shutdown_connections 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.489+0000 7f2446205700 1 -- 172.21.15.177:0/2469563500 wait complete. 2024-04-25T20:58:39.492 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 Processor -- start 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 -- start start 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24401025d0 0x7f2440197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401986f0 0x7f244019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2440108510 con 0x7f24401986f0 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2440108390 con 0x7f24401025d0 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f2446205700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2440108690 con 0x7f2440198270 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:37654/0 (socket says 172.21.15.177:37654) 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 -- 172.21.15.177:0/3264431628 learned_addr learned my addr 172.21.15.177:0/3264431628 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243f7fe700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24401025d0 0x7f2440197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243ffff700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401986f0 0x7f244019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 -- 172.21.15.177:0/3264431628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24401025d0 msgr2=0x7f2440197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:39.493 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24401025d0 0x7f2440197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 -- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401986f0 msgr2=0x7f244019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401986f0 0x7f244019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.490+0000 7f243effd700 1 -- 172.21.15.177:0/3264431628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24401a0150 con 0x7f2440198270 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f243effd700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f243000b780 tx=0x7f243000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430004230 con 0x7f2440198270 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24401a0440 con 0x7f2440198270 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24300043d0 con 0x7f2440198270 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430011b20 con 0x7f2440198270 2024-04-25T20:58:39.494 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.491+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24401a0a00 con 0x7f2440198270 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.492+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2430011cc0 con 0x7f2440198270 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.492+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24400640a0 con 0x7f2440198270 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.492+0000 7f243cff9700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 0x7f242803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.492+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f243004c390 con 0x7f2440198270 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.492+0000 7f243f7fe700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 0x7f242803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:39.495 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.493+0000 7f243f7fe700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 0x7f242803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2440199210 tx=0x7f243401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:39.497 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.497+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2430020500 con 0x7f2440198270 2024-04-25T20:58:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:58:39.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:39.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:39.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:39.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:39 smithi177 ceph-mon[33043]: Reconfiguring daemon mon.b on smithi098 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:39.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:39 smithi098 ceph-mon[32569]: Reconfiguring daemon mon.b on smithi098 2024-04-25T20:58:39.706 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.705+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi098=y;smithi177=z", "target": ["mon-mgr", ""]}) -- 0x7f2440062f10 con 0x7f2428039ae0 2024-04-25T20:58:39.713 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.712+0000 7f243cff9700 1 -- 172.21.15.177:0/3264431628 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2440062f10 con 0x7f2428039ae0 2024-04-25T20:58:39.713 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mgr update... 2024-04-25T20:58:39.715 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 msgr2=0x7f242803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 0x7f242803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2440199210 tx=0x7f243401a040 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 msgr2=0x7f244019d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f243000b780 tx=0x7f243000ba90 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 shutdown_connections 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24401986f0 0x7f244019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2428039ae0 0x7f242803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24401025d0 0x7f2440197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 --2- 172.21.15.177:0/3264431628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2440198270 0x7f244019d2a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 >> 172.21.15.177:0/3264431628 conn(0x7f24400fdb70 msgr2=0x7f24400fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 shutdown_connections 2024-04-25T20:58:39.716 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:58:39.715+0000 7f2446205700 1 -- 172.21.15.177:0/3264431628 wait complete. 2024-04-25T20:58:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:58:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:39.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:39.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:39.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:39 smithi037 ceph-mon[29195]: Reconfiguring daemon mon.b on smithi098 2024-04-25T20:58:40.428 DEBUG:teuthology.orchestra.run.smithi098:mgr.y> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T20:58:40.431 DEBUG:teuthology.orchestra.run.smithi177:mgr.z> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T20:58:40.433 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T20:58:40.433 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T20:58:40.433 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:58:40.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:58:40.454 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 20:57:46.725988598 +0000 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 20:52:42.976304258 +0000 2024-04-25T20:58:40.513 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 20:52:42.976304258 +0000 2024-04-25T20:58:40.514 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 20:52:42.976304258 +0000 2024-04-25T20:58:40.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:58:40.582 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T20:58:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T20:58:40.583 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000243777 s, 2.1 MB/s 2024-04-25T20:58:40.584 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:58:40.644 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-25T20:58:40.701 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:58:40.701 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:40.701 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 20:57:46.759988157 +0000 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 20:52:43.336299429 +0000 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 20:52:43.336299429 +0000 2024-04-25T20:58:40.702 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 20:52:43.336299429 +0000 2024-04-25T20:58:40.702 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:58:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T20:58:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T20:58:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000233187 s, 2.2 MB/s 2024-04-25T20:58:40.770 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:58:40.828 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 20:57:46.788987782 +0000 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 20:52:43.694294627 +0000 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 20:52:43.694294627 +0000 2024-04-25T20:58:40.886 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 20:52:43.694294627 +0000 2024-04-25T20:58:40.887 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi098=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: Saving service mgr spec with placement smithi037=x;smithi098=y;smithi177=z;count:3 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:40.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:40 smithi037 ceph-mon[29195]: Reconfiguring daemon mon.c on smithi177 2024-04-25T20:58:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T20:58:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T20:58:40.951 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00023029 s, 2.2 MB/s 2024-04-25T20:58:40.952 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:58:41.010 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 20:57:46.818987393 +0000 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 20:52:44.048289879 +0000 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 20:52:44.048289879 +0000 2024-04-25T20:58:41.066 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 20:52:44.048289879 +0000 2024-04-25T20:58:41.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:58:41.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi098=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:41.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: Saving service mgr spec with placement smithi037=x;smithi098=y;smithi177=z;count:3 2024-04-25T20:58:41.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:41.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:40 smithi177 ceph-mon[33043]: Reconfiguring daemon mon.c on smithi177 2024-04-25T20:58:41.129 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T20:58:41.129 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T20:58:41.130 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000202998 s, 2.5 MB/s 2024-04-25T20:58:41.131 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:58:41.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi098=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: Saving service mgr spec with placement smithi037=x;smithi098=y;smithi177=z;count:3 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:41.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:40 smithi098 ceph-mon[32569]: Reconfiguring daemon mon.c on smithi177 2024-04-25T20:58:41.187 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-04-25T20:58:41.187 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:58:41.206 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:58:41.207 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-04-25T20:58:41.263 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:58:41.263 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-25 20:58:28.282845477 +0000 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-25 20:52:55.823692684 +0000 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-25 20:52:55.823692684 +0000 2024-04-25T20:58:41.264 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-25 20:52:55.823692684 +0000 2024-04-25T20:58:41.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:58:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-25T20:58:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-25T20:58:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000227626 s, 2.2 MB/s 2024-04-25T20:58:41.331 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:58:41.389 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-04-25T20:58:41.446 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-25 20:58:28.282845477 +0000 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-25 20:52:56.168688722 +0000 2024-04-25T20:58:41.447 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-25 20:52:56.168688722 +0000 2024-04-25T20:58:41.448 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-25 20:52:56.168688722 +0000 2024-04-25T20:58:41.448 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:58:41.512 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-25T20:58:41.512 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-25T20:58:41.512 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000186755 s, 2.7 MB/s 2024-04-25T20:58:41.513 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:58:41.570 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-04-25T20:58:41.629 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:58:41.629 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:41.629 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-25 20:58:28.283845466 +0000 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-25 20:52:56.526684627 +0000 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-25 20:52:56.526684627 +0000 2024-04-25T20:58:41.630 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-25 20:52:56.526684627 +0000 2024-04-25T20:58:41.630 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:58:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-25T20:58:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-25T20:58:41.694 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000249199 s, 2.1 MB/s 2024-04-25T20:58:41.695 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:58:41.753 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-04-25 20:58:28.283845466 +0000 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-04-25 20:52:56.855680863 +0000 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-04-25 20:52:56.855680863 +0000 2024-04-25T20:58:41.809 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: 2024-04-25 20:52:56.855680863 +0000 2024-04-25T20:58:41.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:58:41.872 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-04-25T20:58:41.872 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-04-25T20:58:41.872 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.0001863 s, 2.7 MB/s 2024-04-25T20:58:41.874 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:58:41.930 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2024-04-25T20:58:41.930 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:58:41.947 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:58:41.947 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:42.003 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-25 20:58:18.639330745 +0000 2024-04-25T20:58:42.004 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-25 20:53:00.067122567 +0000 2024-04-25T20:58:42.004 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-25 20:53:00.067122567 +0000 2024-04-25T20:58:42.004 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-25 20:53:00.067122567 +0000 2024-04-25T20:58:42.004 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:58:42.066 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-25T20:58:42.066 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-25T20:58:42.066 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.00025292 s, 2.0 MB/s 2024-04-25T20:58:42.067 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:58:42.123 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2024-04-25T20:58:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:41 smithi098 ceph-mon[32569]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:42.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:42.178 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:58:42.178 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:42.178 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T20:58:42.178 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:42.178 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:42.179 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-25 20:58:18.639330745 +0000 2024-04-25T20:58:42.179 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-25 20:53:00.403113208 +0000 2024-04-25T20:58:42.179 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-25 20:53:00.403113208 +0000 2024-04-25T20:58:42.179 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-25 20:53:00.403113208 +0000 2024-04-25T20:58:42.179 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:58:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:41 smithi037 ceph-mon[29195]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:42.238 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:41 smithi177 ceph-mon[33043]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:42.238 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.238 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.238 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:42.241 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-25T20:58:42.241 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-25T20:58:42.241 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000219159 s, 2.3 MB/s 2024-04-25T20:58:42.242 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:58:42.297 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-25 20:58:18.640330717 +0000 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-25 20:53:00.722104324 +0000 2024-04-25T20:58:42.352 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-25 20:53:00.722104324 +0000 2024-04-25T20:58:42.353 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-25 20:53:00.722104324 +0000 2024-04-25T20:58:42.353 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:58:42.414 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-25T20:58:42.415 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-25T20:58:42.415 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000244159 s, 2.1 MB/s 2024-04-25T20:58:42.416 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:58:42.474 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2024-04-25T20:58:42.530 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2024-04-25 20:58:18.640330717 +0000 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2024-04-25 20:53:01.045095328 +0000 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2024-04-25 20:53:01.045095328 +0000 2024-04-25T20:58:42.531 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: 2024-04-25 20:53:01.045095328 +0000 2024-04-25T20:58:42.531 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2024-04-25T20:58:42.593 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000234049 s, 2.2 MB/s 2024-04-25T20:58:42.594 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:58:42.653 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-25T20:58:42.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:58:42.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:42.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:42.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:42 smithi037 ceph-mon[29195]: Deploying daemon mgr.z on smithi177 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:43.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:42 smithi098 ceph-mon[32569]: Deploying daemon mgr.z on smithi177 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:43.295 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:42 smithi177 ceph-mon[33043]: Deploying daemon mgr.z on smithi177 2024-04-25T20:58:44.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:44 smithi037 ceph-mon[29195]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:44.325 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:44 smithi177 ceph-mon[33043]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:44.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:44 smithi098 ceph-mon[32569]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:44.590 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:44 smithi177 systemd[1]: Started Ceph mgr.z for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:58:44.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:44.589+0000 7f191b330200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:58:44.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:44.771+0000 7f191b330200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:58:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:58:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:58:44.897 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T20:58:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T20:58:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125106 s, 838 MB/s 2024-04-25T20:58:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T20:58:45.175 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:44.871+0000 7f191b330200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:58:45.176 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:45.174+0000 7f191b330200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:58:45.486 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-25T20:58:45.584 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:45.257+0000 7f191b330200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:58:45.730 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:45.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:45.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:45.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:45.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:45 smithi037 ceph-mon[29195]: Deploying daemon mgr.y on smithi098 2024-04-25T20:58:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:45 smithi177 ceph-mon[33043]: Deploying daemon mgr.y on smithi098 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:45.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:45 smithi098 ceph-mon[32569]: Deploying daemon mgr.y on smithi098 2024-04-25T20:58:46.668 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:46 smithi098 systemd[1]: Starting Ceph mgr.y for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T20:58:46.820 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:46.480+0000 7f191b330200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:58:46.820 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:46.649+0000 7f191b330200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:58:47.090 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:46.821+0000 7f191b330200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:46 smithi098 podman[33947]: 2024-04-25 20:58:46.829614062 +0000 UTC m=+0.105365814 container create 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:46 smithi098 podman[33947]: 2024-04-25 20:58:46.742538134 +0000 UTC m=+0.018289871 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 podman[33947]: 2024-04-25 20:58:47.040112859 +0000 UTC m=+0.315864574 container init 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 podman[33947]: 2024-04-25 20:58:47.043022976 +0000 UTC m=+0.318774686 container start 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mgr[33966]: -- 172.21.15.98:0/3693973261 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b146e965a0 con 0x55b146e6a800 2024-04-25T20:58:47.107 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 bash[33947]: 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.289+0000 7fd6398ca700 1 -- 172.21.15.37:0/2052152506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 msgr2=0x7fd63410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.289+0000 7fd6398ca700 1 --2- 172.21.15.37:0/2052152506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd63410ac20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b0a0 tx=0x7fd62c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 -- 172.21.15.37:0/2052152506 shutdown_connections 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 --2- 172.21.15.37:0/2052152506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd63410ac20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 --2- 172.21.15.37:0/2052152506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6341035d0 0x7fd6341059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 --2- 172.21.15.37:0/2052152506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd634105f00 0x7fd6341082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 -- 172.21.15.37:0/2052152506 >> 172.21.15.37:0/2052152506 conn(0x7fd6340fcf40 msgr2=0x7fd6340ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 -- 172.21.15.37:0/2052152506 shutdown_connections 2024-04-25T20:58:47.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 -- 172.21.15.37:0/2052152506 wait complete. 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.290+0000 7fd6398ca700 1 Processor -- start 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 -- start start 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6341035d0 0x7fd634102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410e530 con 0x7fd634108830 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410e3b0 con 0x7fd634105f00 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6398ca700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd63410e6b0 con 0x7fd6341035d0 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6327fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6327fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:49592/0 (socket says 172.21.15.37:49592) 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6327fc700 1 -- 172.21.15.37:0/3309004209 learned_addr learned my addr 172.21.15.37:0/3309004209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45230/0 (socket says 172.21.15.37:45230) 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd632ffd700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6341035d0 0x7fd634102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 -- 172.21.15.37:0/3309004209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6341035d0 msgr2=0x7fd634102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6341035d0 0x7fd634102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 -- 172.21.15.37:0/3309004209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 msgr2=0x7fd634101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6337fe700 1 -- 172.21.15.37:0/3309004209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd634102370 con 0x7fd634108830 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.291+0000 7fd6327fc700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6337fe700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b070 tx=0x7fd62c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c021070 con 0x7fd634108830 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6341a2d30 con 0x7fd634108830 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd62c007e30 con 0x7fd634108830 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c004730 con 0x7fd634108830 2024-04-25T20:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.292+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6341a3190 con 0x7fd634108830 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.293+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd62c01d450 con 0x7fd634108830 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.293+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63402df80 con 0x7fd634108830 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.293+0000 7fd6388c8700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 0x7fd61c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.293+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd62c00f020 con 0x7fd634108830 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.293+0000 7fd632ffd700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 0x7fd61c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.294+0000 7fd632ffd700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 0x7fd61c03bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd624009c60 tx=0x7fd624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.299+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd62c02a020 con 0x7fd634108830 2024-04-25T20:58:47.387 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 systemd[1]: Started Ceph mgr.y for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T20:58:47.387 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:47.200+0000 7f8ba4f67200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:58:47.388 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:47.386+0000 7f8ba4f67200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:58:47.408 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:47.107+0000 7f191b330200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:58:47.408 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:47.199+0000 7f191b330200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:58:47.408 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:47.407+0000 7f191b330200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:58:47.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:47.515+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd634062e80 con 0x7fd61c039890 2024-04-25T20:58:47.667 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:47.494+0000 7f8ba4f67200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:58:47.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:47.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:47.609+0000 7f191b330200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:58:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:58:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:47.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:47.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:58:47.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:48.168 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:47.811+0000 7f8ba4f67200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:58:48.168 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:47.900+0000 7f8ba4f67200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:58:48.590 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:48.248+0000 7f191b330200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:58:48.730 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:48 smithi037 ceph-mon[29195]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:48.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:48 smithi098 ceph-mon[32569]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:48.985 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:48 smithi177 ceph-mon[33043]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:58:48.985 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:48.677+0000 7f191b330200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:58:48.985 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:48.780+0000 7f191b330200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:58:48.985 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:48.984+0000 7f191b330200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:58:49.417 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:49.119+0000 7f8ba4f67200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:58:49.418 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:49.286+0000 7f8ba4f67200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:49.453+0000 7f8ba4f67200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:49.735+0000 7f8ba4f67200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.737 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T20:58:49.738 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:49.515+0000 7f191b330200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:58:49.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:49.607+0000 7f191b330200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:58:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T20:58:49.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:58:49.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:49.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: Reconfiguring daemon mgr.x on smithi037 2024-04-25T20:58:49.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:50.027 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:49.826+0000 7f8ba4f67200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:58:50.027 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:50.026+0000 7f8ba4f67200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:58:50.190 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:49.849+0000 7f191b330200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:58:50.190 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:49.981+0000 7f191b330200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:58:50.190 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:50.075+0000 7f191b330200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:58:50.190 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:50.188+0000 7f191b330200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:58:50.418 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:50.215+0000 7f8ba4f67200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:58:50.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:50.491+0000 7f191b330200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:58:50.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]: dispatch 2024-04-25T20:58:50.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]': finished 2024-04-25T20:58:50.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-mon[33043]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:58:50.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:50 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:58:50.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:50 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]: dispatch 2024-04-25T20:58:50.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:50 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]': finished 2024-04-25T20:58:50.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:50 smithi037 ceph-mon[29195]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:58:50.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:58:51.167 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:50.804+0000 7f8ba4f67200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:58:51.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]: dispatch 2024-04-25T20:58:51.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1057888347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a581ae-b05a-4399-8fc9-4fe21b79d2bf"}]': finished 2024-04-25T20:58:51.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-mon[32569]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:58:51.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:50 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:58:51.340 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:51.087+0000 7f191b330200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:58:51.340 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T20:58:51.302+0000 7f191b330200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:58:51.515 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:51.220+0000 7f8ba4f67200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:58:51.516 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:51.315+0000 7f8ba4f67200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:58:51.516 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:51.514+0000 7f8ba4f67200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:58:51.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:51.701+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fd62c005040 con 0x7fd634108830 2024-04-25T20:58:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/9023354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: Standby manager daemon z started 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:58:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:51 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:51.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/9023354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: Standby manager daemon z started 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:58:51.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:51 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:52.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:52.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/9023354' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:58:52.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: Standby manager daemon z started 2024-04-25T20:58:52.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:58:52.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:52.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:58:52.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:51 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3279115813' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:52.356 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.028+0000 7f8ba4f67200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:58:52.356 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.117+0000 7f8ba4f67200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:58:52.667 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.355+0000 7f8ba4f67200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:58:52.668 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.484+0000 7f8ba4f67200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:58:52.668 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.577+0000 7f8ba4f67200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:58:52.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:52 smithi037 ceph-mon[29195]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T20:58:52.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:52 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:58:52.987 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.690+0000 7f8ba4f67200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:58:52.987 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-mon[32569]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T20:58:52.987 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:58:53.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:52 smithi177 ceph-mon[33043]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T20:58:53.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:52 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:58:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:53 smithi098 ceph-mon[32569]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:53.918 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:52.986+0000 7f8ba4f67200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:58:53.918 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:53.575+0000 7f8ba4f67200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:58:53.918 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 20:58:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T20:58:53.787+0000 7f8ba4f67200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:58:53.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:53 smithi037 ceph-mon[29195]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:54.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:53 smithi177 ceph-mon[33043]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:54.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:58:54.722+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd62c01db60 con 0x7fd634108830 2024-04-25T20:58:54.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:54 smithi037 ceph-mon[29195]: Standby manager daemon y started 2024-04-25T20:58:54.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:54 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:58:54.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:54 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:54.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:54 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:58:54.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:54 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:54 smithi177 ceph-mon[33043]: Standby manager daemon y started 2024-04-25T20:58:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:54 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:58:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:54 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:54 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:58:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:54 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:55.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:54 smithi098 ceph-mon[32569]: Standby manager daemon y started 2024-04-25T20:58:55.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:54 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:58:55.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:54 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:58:55.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:54 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:58:55.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:54 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1554053892' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:58:55.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:55 smithi037 ceph-mon[29195]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:55.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:55 smithi037 ceph-mon[29195]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-04-25T20:58:55.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:58:56.013 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:55 smithi098 ceph-mon[32569]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:56.013 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:55 smithi098 ceph-mon[32569]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-04-25T20:58:56.013 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:58:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:55 smithi177 ceph-mon[33043]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:55 smithi177 ceph-mon[33043]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-04-25T20:58:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:58:57.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:57.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:57.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:57.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:57.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:57.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:57.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:57.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:57.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:58:58.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:57 smithi037 ceph-mon[29195]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:58.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:57 smithi177 ceph-mon[33043]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:58.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:57 smithi098 ceph-mon[32569]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:58:59.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:58:59.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:58:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:58:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:59.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:58:59.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:00 smithi177 ceph-mon[33043]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:00 smithi177 ceph-mon[33043]: Deploying daemon osd.0 on smithi037 2024-04-25T20:59:00.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:59 smithi037 ceph-mon[29195]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:00.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:58:59 smithi037 ceph-mon[29195]: Deploying daemon osd.0 on smithi037 2024-04-25T20:59:00.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:59 smithi098 ceph-mon[32569]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:00.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:58:59 smithi098 ceph-mon[32569]: Deploying daemon osd.0 on smithi037 2024-04-25T20:59:02.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:02 smithi037 ceph-mon[29195]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:02.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:02 smithi177 ceph-mon[33043]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:02.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:02 smithi098 ceph-mon[32569]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:04.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:04 smithi037 ceph-mon[29195]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:04.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:04.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:04 smithi177 ceph-mon[33043]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:04.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:04.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:04 smithi098 ceph-mon[32569]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:04.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:04.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:06.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:06.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:06.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:06.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:06.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:06.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:07.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.126+0000 7fd6388c8700 1 -- 172.21.15.37:0/3309004209 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd634062e80 con 0x7fd61c039890 2024-04-25T20:59:07.129 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-04-25T20:59:07.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 msgr2=0x7fd61c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 0x7fd61c03bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd624009c60 tx=0x7fd624009260 comp rx=0 tx=0).stop 2024-04-25T20:59:07.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 msgr2=0x7fd634101ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:07.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b070 tx=0x7fd62c004d00 comp rx=0 tx=0).stop 2024-04-25T20:59:07.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 shutdown_connections 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd634108830 0x7fd634101ac0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6341035d0 0x7fd634102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd61c039890 0x7fd61c03bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 --2- 172.21.15.37:0/3309004209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd634105f00 0x7fd634101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 >> 172.21.15.37:0/3309004209 conn(0x7fd6340fcf40 msgr2=0x7fd634104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 shutdown_connections 2024-04-25T20:59:07.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T20:59:07.129+0000 7fd6398ca700 1 -- 172.21.15.37:0/3309004209 wait complete. 2024-04-25T20:59:07.806 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T20:59:07.808 INFO:tasks.cephadm:Deploying osd.1 on smithi098 with /dev/vg_nvme/lv_4... 2024-04-25T20:59:07.808 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:59:08.050 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:59:08.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:08 smithi037 ceph-mon[29195]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:08.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:08 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:08 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:08 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:08.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:08 smithi037 ceph-mon[29195]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:59:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:08 smithi098 ceph-mon[32569]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:08 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:08 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:08 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:08.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:08 smithi098 ceph-mon[32569]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:59:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:08 smithi177 ceph-mon[33043]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:08 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:08 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:08 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:08 smithi177 ceph-mon[33043]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:59:09.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-mon[29195]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:59:09.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-mon[29195]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:59:09.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-mon[29195]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T20:59:09.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:09.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-mon[29195]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:09.236 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0[43821]: 2024-04-25T20:59:09.109+0000 7efded9fc700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T20:59:09.236 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 20:59:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0[43821]: 2024-04-25T20:59:09.116+0000 7efde97ef700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:59:09.328 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:09 smithi098 ceph-mon[32569]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:59:09.328 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:09 smithi098 ceph-mon[32569]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:59:09.328 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:09 smithi098 ceph-mon[32569]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T20:59:09.328 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:09.328 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:09 smithi098 ceph-mon[32569]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:09 smithi177 ceph-mon[33043]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:59:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:09 smithi177 ceph-mon[33043]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:59:09.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:09 smithi177 ceph-mon[33043]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T20:59:09.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:09.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:09 smithi177 ceph-mon[33043]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:09.877 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:59:09.877 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:59:09.877 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-25T20:59:09.877 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-25T20:59:09.877 INFO:teuthology.orchestra.run.smithi098.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113167 s, 927 MB/s 2024-04-25T20:59:09.878 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 2024-04-25T20:59:09.878 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-25T20:59:10.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:10 smithi098 ceph-mon[32569]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T20:59:10.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:10 smithi098 ceph-mon[32569]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:59:10.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:10 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:10 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.467 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:10 smithi037 ceph-mon[29195]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T20:59:10.467 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:10 smithi037 ceph-mon[29195]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:59:10.467 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:10 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.467 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:10 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_4 2024-04-25T20:59:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:10 smithi177 ceph-mon[33043]: from='osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T20:59:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:10 smithi177 ceph-mon[33043]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:59:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:10 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:10 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:10.724 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] boot 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi037 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:11.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T20:59:11.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:11.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:11.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:59:11.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:59:11.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] boot 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi037 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T20:59:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T20:59:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: osd.0 [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] boot 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi037 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 17093M 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:11.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:11.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3580292143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc104630 msgr2=0x7fb0cc10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3580292143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc104630 0x7fb0cc10aee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b0d0 tx=0x7fb0c400b3e0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3580292143 shutdown_connections 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3580292143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc104630 0x7fb0cc10aee0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3580292143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc102a90 0x7fb0cc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3580292143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc103c90 0x7fb0cc1040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3580292143 >> 172.21.15.98:0/3580292143 conn(0x7fb0cc0fe030 msgr2=0x7fb0cc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3580292143 shutdown_connections 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3580292143 wait complete. 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 Processor -- start 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.315+0000 7fb0d2fde700 1 -- start start 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc103c90 0x7fb0cc199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 0x7fb0cc1a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc10e510 con 0x7fb0cc103c90 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc10e390 con 0x7fb0cc102a90 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc10e690 con 0x7fb0cc104630 2024-04-25T20:59:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46516/0 (socket says 172.21.15.98:46516) 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 -- 172.21.15.98:0/3782573976 learned_addr learned my addr 172.21.15.98:0/3782573976 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0cbfff700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc103c90 0x7fb0cc199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d157b700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 0x7fb0cc1a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 -- 172.21.15.98:0/3782573976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 msgr2=0x7fb0cc1a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 0x7fb0cc1a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 -- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc103c90 msgr2=0x7fb0cc199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc103c90 0x7fb0cc199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 -- 172.21.15.98:0/3782573976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1a07a0 con 0x7fb0cc102a90 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d157b700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 0x7fb0cc1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d0d7a700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000e970 tx=0x7fb0c000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000cbd0 con 0x7fb0cc102a90 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc1a0a90 con 0x7fb0cc102a90 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c00044d0 con 0x7fb0cc102a90 2024-04-25T20:59:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.316+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003df0 con 0x7fb0cc102a90 2024-04-25T20:59:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.317+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1a1020 con 0x7fb0cc102a90 2024-04-25T20:59:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.318+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc069730 con 0x7fb0cc102a90 2024-04-25T20:59:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.319+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb0c0004000 con 0x7fb0cc102a90 2024-04-25T20:59:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.319+0000 7fb0c9ffb700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 0x7fb0bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.319+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb0c00ceb90 con 0x7fb0cc102a90 2024-04-25T20:59:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.319+0000 7fb0cbfff700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 0x7fb0bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.320+0000 7fb0cbfff700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 0x7fb0bc0a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8007be0 tx=0x7fb0b80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:12.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.324+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0c00042f0 con 0x7fb0cc102a90 2024-04-25T20:59:12.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:12 smithi098 ceph-mon[32569]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:59:12.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:12 smithi037 ceph-mon[29195]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:59:12.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:12.529+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb0cc062e80 con 0x7fb0bc0a3bb0 2024-04-25T20:59:12.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:12 smithi177 ceph-mon[33043]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:59:13.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:13 smithi098 ceph-mon[32569]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:13 smithi098 ceph-mon[32569]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:13.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:13 smithi037 ceph-mon[29195]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:13.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:13 smithi037 ceph-mon[29195]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:13.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:13.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:13.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:13 smithi177 ceph-mon[33043]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:13 smithi177 ceph-mon[33043]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:13.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:15.392 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:15 smithi177 ceph-mon[33043]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:15.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:15 smithi098 ceph-mon[32569]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:15.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:15 smithi037 ceph-mon[29195]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:16.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/3687852906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]': finished 2024-04-25T20:59:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:59:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:16 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/2615244662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/3687852906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]': finished 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:16.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:16 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/2615244662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/3687852906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]: dispatch 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d16bca01-f418-416e-9d6a-57b627ccd028"}]': finished 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:16.582 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:16 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/2615244662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:17.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:17 smithi177 ceph-mon[33043]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:17 smithi098 ceph-mon[32569]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:17.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:17 smithi037 ceph-mon[29195]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:19.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:19 smithi098 ceph-mon[32569]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:19 smithi177 ceph-mon[33043]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:19.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:19 smithi037 ceph-mon[29195]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:21.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:21 smithi098 ceph-mon[32569]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:21 smithi037 ceph-mon[29195]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:21 smithi177 ceph-mon[33043]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:22.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:59:22.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:22.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:59:22.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:22.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:59:22.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:23.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:23 smithi098 ceph-mon[32569]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:23.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:23 smithi098 ceph-mon[32569]: Deploying daemon osd.1 on smithi098 2024-04-25T20:59:23.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:23 smithi177 ceph-mon[33043]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:23.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:23 smithi177 ceph-mon[33043]: Deploying daemon osd.1 on smithi098 2024-04-25T20:59:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:23 smithi037 ceph-mon[29195]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:23 smithi037 ceph-mon[29195]: Deploying daemon osd.1 on smithi098 2024-04-25T20:59:25.501 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:25 smithi098 ceph-mon[32569]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:25.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:25 smithi177 ceph-mon[33043]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:25 smithi037 ceph-mon[29195]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:27 smithi177 ceph-mon[33043]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:27 smithi098 ceph-mon[32569]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:27 smithi037 ceph-mon[29195]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:28.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:28.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.787 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:29.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:29 smithi098 ceph-mon[32569]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:29.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:29 smithi037 ceph-mon[29195]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:29 smithi177 ceph-mon[33043]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.096+0000 7fb0c9ffb700 1 -- 172.21.15.98:0/3782573976 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb0cc062e80 con 0x7fb0bc0a3bb0 2024-04-25T20:59:31.096 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 1 on host 'smithi098' 2024-04-25T20:59:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 msgr2=0x7fb0bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 0x7fb0bc0a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8007be0 tx=0x7fb0b80073d0 comp rx=0 tx=0).stop 2024-04-25T20:59:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 msgr2=0x7fb0cc198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000e970 tx=0x7fb0c000ec80 comp rx=0 tx=0).stop 2024-04-25T20:59:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 shutdown_connections 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc104630 0x7fb0cc1a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb0bc0a3bb0 0x7fb0bc0a6070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0cc102a90 0x7fb0cc198be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 --2- 172.21.15.98:0/3782573976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0cc103c90 0x7fb0cc199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 >> 172.21.15.98:0/3782573976 conn(0x7fb0cc0fe030 msgr2=0x7fb0cc106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 shutdown_connections 2024-04-25T20:59:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T20:59:31.098+0000 7fb0d2fde700 1 -- 172.21.15.98:0/3782573976 wait complete. 2024-04-25T20:59:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:31.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:31.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:31.761 DEBUG:teuthology.orchestra.run.smithi098:osd.1> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T20:59:31.763 INFO:tasks.cephadm:Deploying osd.2 on smithi177 with /dev/vg_nvme/lv_4... 2024-04-25T20:59:31.763 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:59:32.017 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:59:32.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:32 smithi177 ceph-mon[33043]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:32.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:32 smithi177 ceph-mon[33043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:32 smithi098 ceph-mon[32569]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:32 smithi098 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:32.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:32 smithi037 ceph-mon[29195]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:32.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:32 smithi037 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:33 smithi177 ceph-mon[33043]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.669 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1[37171]: 2024-04-25T20:59:33.327+0000 7fd135a56700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T20:59:33.669 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 20:59:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1[37171]: 2024-04-25T20:59:33.331+0000 7fd131849700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: from='osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:33 smithi037 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T20:59:33.897 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:59:33.898 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:59:33.898 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-04-25T20:59:33.898 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-04-25T20:59:33.898 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111819 s, 938 MB/s 2024-04-25T20:59:33.898 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-04-25T20:59:34.501 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:34 smithi177 ceph-mon[33043]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T20:59:34.501 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:34 smithi177 ceph-mon[33043]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:59:34.501 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.501 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.530 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_4 2024-04-25T20:59:34.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:34 smithi098 ceph-mon[32569]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T20:59:34.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:34 smithi098 ceph-mon[32569]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:59:34.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:34 smithi037 ceph-mon[29195]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T20:59:34.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:34 smithi037 ceph-mon[29195]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:59:34.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:34.771 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:59:35.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T20:59:35.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T20:59:35.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:35.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] boot 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi098 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 17088M 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:35.449 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] boot 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi098 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 17088M 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:35.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: osd.1 [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] boot 2024-04-25T20:59:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi098 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 17088M 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:35.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:36.247 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 -- 172.21.15.177:0/922542161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103550 msgr2=0x7f5424109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:36.247 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 --2- 172.21.15.177:0/922542161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103550 0x7f5424109e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f541c00b0a0 tx=0x7f541c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.247 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 -- 172.21.15.177:0/922542161 shutdown_connections 2024-04-25T20:59:36.247 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 --2- 172.21.15.177:0/922542161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103550 0x7f5424109e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 --2- 172.21.15.177:0/922542161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f5424101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 --2- 172.21.15.177:0/922542161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424102bb0 0x7f5424103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 -- 172.21.15.177:0/922542161 >> 172.21.15.177:0/922542161 conn(0x7f54240fcf70 msgr2=0x7f54240ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 -- 172.21.15.177:0/922542161 shutdown_connections 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.246+0000 7f54290c6700 1 -- 172.21.15.177:0/922542161 wait complete. 2024-04-25T20:59:36.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 Processor -- start 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 -- start start 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f542406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 0x7f542419fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d4a0 con 0x7f5424103550 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d320 con 0x7f54241019b0 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f54290c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f542410d620 con 0x7f5424102bb0 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f542259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f542259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:41456/0 (socket says 172.21.15.177:41456) 2024-04-25T20:59:36.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.247+0000 7f542259c700 1 -- 172.21.15.177:0/622118658 learned_addr learned my addr 172.21.15.177:0/622118658 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f5422d9d700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f542406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542359e700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 0x7f542419fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 -- 172.21.15.177:0/622118658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 msgr2=0x7f542406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f542406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 -- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 msgr2=0x7f542419fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 0x7f542419fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 -- 172.21.15.177:0/622118658 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54241a08c0 con 0x7f5424102bb0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542359e700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 0x7f542419fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f5422d9d700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f542406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f542259c700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f541800c920 tx=0x7f541800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418007b00 con 0x7f5424102bb0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54241a0bb0 con 0x7f5424102bb0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5418004ce0 con 0x7f5424102bb0 2024-04-25T20:59:36.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54241a1140 con 0x7f5424102bb0 2024-04-25T20:59:36.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.248+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418005650 con 0x7f5424102bb0 2024-04-25T20:59:36.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.249+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5424069730 con 0x7f5424102bb0 2024-04-25T20:59:36.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.250+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f5418007ca0 con 0x7f5424102bb0 2024-04-25T20:59:36.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.251+0000 7f540bfff700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 0x7f540c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:36.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.251+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f54180ceb20 con 0x7f5424102bb0 2024-04-25T20:59:36.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.251+0000 7f5422d9d700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 0x7f540c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:36.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.252+0000 7f5422d9d700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 0x7f540c0a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5414005950 tx=0x7f5414005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:36.256 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.256+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5418014030 con 0x7f5424102bb0 2024-04-25T20:59:36.464 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:36.463+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5424062e80 con 0x7f540c0a3bb0 2024-04-25T20:59:37.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:36 smithi098 ceph-mon[32569]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:59:37.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:37.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:37.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:36 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:37.175 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:36 smithi177 ceph-mon[33043]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:59:37.175 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:37.175 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:37.176 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:36 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:37.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:36 smithi037 ceph-mon[29195]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:59:37.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:59:37.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:59:37.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:36 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:37.990 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:37 smithi177 ceph-mon[33043]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:37.990 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:37 smithi177 ceph-mon[33043]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:37 smithi098 ceph-mon[32569]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:38.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:37 smithi098 ceph-mon[32569]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:38.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:37 smithi037 ceph-mon[29195]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:38.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:37 smithi037 ceph-mon[29195]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:59:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/592506597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]': finished 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:40.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:39 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/2322620653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/592506597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]': finished 2024-04-25T20:59:40.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:59:40.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:40.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:39 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/2322620653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/592506597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]: dispatch 2024-04-25T20:59:40.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "25aa2e48-4132-499e-8958-d7f5b4638000"}]': finished 2024-04-25T20:59:40.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:59:40.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:40.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:39 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/2322620653' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:59:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:41 smithi098 ceph-mon[32569]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:41 smithi037 ceph-mon[29195]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:42.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:41 smithi177 ceph-mon[33043]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:44.126 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:43 smithi177 ceph-mon[33043]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:43 smithi098 ceph-mon[32569]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:44.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:43 smithi037 ceph-mon[29195]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:46.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:45 smithi177 ceph-mon[33043]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:45 smithi098 ceph-mon[32569]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:46.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:45 smithi037 ceph-mon[29195]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:47.139 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:59:47.139 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:47.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:59:47.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:47.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:59:47.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:48.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:47 smithi177 ceph-mon[33043]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:48.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:47 smithi177 ceph-mon[33043]: Deploying daemon osd.2 on smithi177 2024-04-25T20:59:48.162 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:47 smithi098 ceph-mon[32569]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:48.162 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:47 smithi098 ceph-mon[32569]: Deploying daemon osd.2 on smithi177 2024-04-25T20:59:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:47 smithi037 ceph-mon[29195]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:47 smithi037 ceph-mon[29195]: Deploying daemon osd.2 on smithi177 2024-04-25T20:59:50.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:49 smithi177 ceph-mon[33043]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:49 smithi098 ceph-mon[32569]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:49 smithi037 ceph-mon[29195]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:51 smithi098 ceph-mon[32569]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:52.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:51 smithi037 ceph-mon[29195]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:52.301 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:51 smithi177 ceph-mon[33043]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:52.301 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.302 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:52.302 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:54.129 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:53 smithi177 ceph-mon[33043]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:53 smithi098 ceph-mon[32569]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:54.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:53 smithi037 ceph-mon[29195]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:55.587 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.586+0000 7f540bfff700 1 -- 172.21.15.177:0/622118658 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5424062e80 con 0x7f540c0a3bb0 2024-04-25T20:59:55.587 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 2 on host 'smithi177' 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 msgr2=0x7f540c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 0x7f540c0a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5414005950 tx=0x7f5414005f50 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 msgr2=0x7f542419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f541800c920 tx=0x7f541800cce0 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 shutdown_connections 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5424103550 0x7f542419fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f540c0a3bb0 0x7f540c0a6070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f54241019b0 0x7f542406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 --2- 172.21.15.177:0/622118658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424102bb0 0x7f542419d9a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 >> 172.21.15.177:0/622118658 conn(0x7f54240fcf70 msgr2=0x7f5424105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 shutdown_connections 2024-04-25T20:59:55.591 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T20:59:55.590+0000 7f54290c6700 1 -- 172.21.15.177:0/622118658 wait complete. 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.944 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:55.945 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:55.945 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:55 smithi177 ceph-mon[33043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:55.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:55.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:55.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:55.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:55 smithi037 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:56.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:55 smithi098 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:59:56.252 DEBUG:teuthology.orchestra.run.smithi177:osd.2> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T20:59:56.254 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-25T20:59:56.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:59:56.509 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:57.740 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-mon[33043]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:57.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:57.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:59:57.817 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:59:57.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:57.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:57.818 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:57 smithi098 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: from='osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:57.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:57 smithi037 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T20:59:58.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2[38222]: 2024-04-25T20:59:57.975+0000 7fc149ad4700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T20:59:58.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 20:59:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2[38222]: 2024-04-25T20:59:57.979+0000 7fc1418bf700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130061 s, 806 MB/s 2024-04-25T20:59:58.629 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T20:59:58.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T20:59:58.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:59:58.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: from='osd.2 ' entity='osd.2' 2024-04-25T20:59:58.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T20:59:58.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:59:58.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: from='osd.2 ' entity='osd.2' 2024-04-25T20:59:58.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T20:59:58.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:59:58.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:58.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: from='osd.2 ' entity='osd.2' 2024-04-25T20:59:58.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.301 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-25T20:59:59.548 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] boot 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:59.780 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.780 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:59.780 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:59.780 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 20:59:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] boot 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:59.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:59.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:59.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 20:59:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T20:59:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: osd.2 [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] boot 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:59.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 20:59:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:00.812 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:00 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi177 2024-04-25T21:00:00.812 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:00 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 17088M 2024-04-25T21:00:00.813 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:00 smithi037 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T21:00:00.813 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:00 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:00.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:00 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi177 2024-04-25T21:00:00.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:00 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 17088M 2024-04-25T21:00:00.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:00 smithi177 ceph-mon[33043]: overall HEALTH_OK 2024-04-25T21:00:00.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:00 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:00.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:00 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi177 2024-04-25T21:00:00.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:00 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 17088M 2024-04-25T21:00:00.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:00 smithi098 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T21:00:00.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:00 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 -- 172.21.15.37:0/2215572009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694102bc0 msgr2=0x7f6694103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 --2- 172.21.15.37:0/2215572009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694102bc0 0x7f6694103020 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66800099c0 tx=0x7f6680009cd0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 -- 172.21.15.37:0/2215572009 shutdown_connections 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 --2- 172.21.15.37:0/2215572009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6694103560 0x7f6694109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 --2- 172.21.15.37:0/2215572009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66941019c0 0x7f6694101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 --2- 172.21.15.37:0/2215572009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694102bc0 0x7f6694103020 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 -- 172.21.15.37:0/2215572009 >> 172.21.15.37:0/2215572009 conn(0x7f66940fcf60 msgr2=0x7f66940ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.177+0000 7f669ae15700 1 -- 172.21.15.37:0/2215572009 shutdown_connections 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 -- 172.21.15.37:0/2215572009 wait complete. 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 Processor -- start 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 -- start start 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66941019c0 0x7f669406bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 0x7f669419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f669410d450 con 0x7f6694103560 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f669410d2d0 con 0x7f66941019c0 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.178+0000 7f669ae15700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f669410d5d0 con 0x7f6694102bc0 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33170/0 (socket says 172.21.15.37:33170) 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 -- 172.21.15.37:0/848435392 learned_addr learned my addr 172.21.15.37:0/848435392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6698bb1700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66941019c0 0x7f669406bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6693fff700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 0x7f669419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 -- 172.21.15.37:0/848435392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 msgr2=0x7f669419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 0x7f669419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 -- 172.21.15.37:0/848435392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66941019c0 msgr2=0x7f669406bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66941019c0 0x7f669406bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 -- 172.21.15.37:0/848435392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941a2a80 con 0x7f6694103560 2024-04-25T21:00:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6693fff700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 0x7f669419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f66993b2700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f668c00ada0 tx=0x7f668c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c017020 con 0x7f6694103560 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66941a2d70 con 0x7f6694103560 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668c00ce30 con 0x7f6694103560 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.179+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c005350 con 0x7f6694103560 2024-04-25T21:00:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.180+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66941a3280 con 0x7f6694103560 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.181+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694069730 con 0x7f6694103560 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.182+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f668c00c980 con 0x7f6694103560 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.182+0000 7f6691ffb700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 0x7f66840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.183+0000 7f6698bb1700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 0x7f66840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.183+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f668c0cec80 con 0x7f6694103560 2024-04-25T21:00:01.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.183+0000 7f6698bb1700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 0x7f66840a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6688005950 tx=0x7f668800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.187+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f668c021020 con 0x7f6694103560 2024-04-25T21:00:01.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:01.404+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6694062e80 con 0x7f66840a3bb0 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:01.668 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:01.820 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:01.820 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.821 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.821 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41826]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:01.821 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:01.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T21:00:01.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:00:01.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:01.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:01.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41020]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41020]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41023]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:01.918 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.919 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41023]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.919 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41023]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.985 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:01.985 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.985 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48581]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.986 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48581]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.986 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:01.986 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.986 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48584]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.986 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48584]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48587]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48587]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:01.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:01.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48597]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:01.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:01 smithi037 sudo[48597]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:02.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:02.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:02.090 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41829]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41832]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41832]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41832]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41832]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41835]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41835]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41835]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:02.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:01 smithi177 sudo[41835]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41026]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T21:00:02.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:01 smithi098 sudo[41029]: pam_unix(sudo:session): session closed for user root 2024-04-25T21:00:02.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:02 smithi037 ceph-mon[29195]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:02.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T21:00:02.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:02 smithi177 ceph-mon[33043]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:02.857+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f668c098a80 con 0x7f6694103560 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:00:02.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:02 smithi098 ceph-mon[32569]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T21:00:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:03 smithi037 ceph-mon[29195]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:03 smithi037 ceph-mon[29195]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T21:00:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:03 smithi037 ceph-mon[29195]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T21:00:03.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:03 smithi177 ceph-mon[33043]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:03.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:03 smithi177 ceph-mon[33043]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T21:00:03.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:03 smithi177 ceph-mon[33043]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T21:00:03.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:03 smithi098 ceph-mon[32569]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:03.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:03 smithi098 ceph-mon[32569]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T21:00:03.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:03 smithi098 ceph-mon[32569]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T21:00:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:04 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]: dispatch 2024-04-25T21:00:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:04 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]': finished 2024-04-25T21:00:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:04 smithi177 ceph-mon[33043]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T21:00:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:04.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:04 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]: dispatch 2024-04-25T21:00:04.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:04 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]': finished 2024-04-25T21:00:04.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:04 smithi098 ceph-mon[32569]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T21:00:04.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:04 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]: dispatch 2024-04-25T21:00:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:04 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4186884959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd7eb40c-16dd-48be-b06b-2278b4ea4333"}]': finished 2024-04-25T21:00:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:04 smithi037 ceph-mon[29195]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T21:00:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:05.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:05 smithi177 ceph-mon[33043]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:05.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:05 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1821629787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:05 smithi098 ceph-mon[32569]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:05 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1821629787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:05 smithi037 ceph-mon[29195]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:05 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1821629787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:07.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:07 smithi177 ceph-mon[33043]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:07 smithi098 ceph-mon[32569]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:07 smithi037 ceph-mon[29195]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:09.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:09 smithi177 ceph-mon[33043]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:09.866 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:09 smithi037 ceph-mon[29195]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:09.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:09 smithi098 ceph-mon[32569]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:11.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:11 smithi177 ceph-mon[33043]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:11.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:11 smithi098 ceph-mon[32569]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:11 smithi037 ceph-mon[29195]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:12.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T21:00:12.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:12.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T21:00:12.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T21:00:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:13.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:13 smithi037 ceph-mon[29195]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:13.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:13 smithi037 ceph-mon[29195]: Deploying daemon osd.3 on smithi037 2024-04-25T21:00:13.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:13 smithi098 ceph-mon[32569]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:13.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:13 smithi098 ceph-mon[32569]: Deploying daemon osd.3 on smithi037 2024-04-25T21:00:14.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:13 smithi177 ceph-mon[33043]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:14.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:13 smithi177 ceph-mon[33043]: Deploying daemon osd.3 on smithi037 2024-04-25T21:00:15.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:15 smithi037 ceph-mon[29195]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:15.892 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:15 smithi177 ceph-mon[33043]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:15 smithi098 ceph-mon[32569]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:17.615 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:17 smithi037 ceph-mon[29195]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:17 smithi098 ceph-mon[32569]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:17 smithi177 ceph-mon[33043]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:19.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:19 smithi037 ceph-mon[29195]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:19.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:19.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:19 smithi098 ceph-mon[32569]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:19.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:19.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:19 smithi177 ceph-mon[33043]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:19.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:19.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:21 smithi037 ceph-mon[29195]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:21 smithi098 ceph-mon[32569]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:21.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:21 smithi177 ceph-mon[33043]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:21.869 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-04-25T21:00:21.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.867+0000 7f6691ffb700 1 -- 172.21.15.37:0/848435392 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6694062e80 con 0x7f66840a3bb0 2024-04-25T21:00:21.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.869+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 msgr2=0x7f66840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.869+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 0x7f66840a6070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6688005950 tx=0x7f668800a300 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 msgr2=0x7f66941a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f668c00ada0 tx=0x7f668c00c330 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 shutdown_connections 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6694103560 0x7f66941a22a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f66840a3bb0 0x7f66840a6070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66941019c0 0x7f669406bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 --2- 172.21.15.37:0/848435392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6694102bc0 0x7f669419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 >> 172.21.15.37:0/848435392 conn(0x7f66940fcf60 msgr2=0x7f6694105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:21.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 shutdown_connections 2024-04-25T21:00:21.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:00:21.870+0000 7f669ae15700 1 -- 172.21.15.37:0/848435392 wait complete. 2024-04-25T21:00:22.560 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T21:00:22.562 INFO:tasks.cephadm:Deploying osd.4 on smithi098 with /dev/vg_nvme/lv_3... 2024-04-25T21:00:22.562 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T21:00:22.805 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:00:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:22 smithi037 ceph-mon[29195]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T21:00:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:22 smithi098 ceph-mon[32569]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T21:00:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:22 smithi177 ceph-mon[33043]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T21:00:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:23 smithi098 ceph-mon[32569]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:23.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:23 smithi098 ceph-mon[32569]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T21:00:23.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:23 smithi098 ceph-mon[32569]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T21:00:23.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:23 smithi098 ceph-mon[32569]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:00:23.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:23 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:23.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:23 smithi037 ceph-mon[29195]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:23.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:23 smithi037 ceph-mon[29195]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T21:00:23.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:23 smithi037 ceph-mon[29195]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T21:00:23.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:23 smithi037 ceph-mon[29195]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:00:23.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:23 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:24.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:23 smithi177 ceph-mon[33043]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:24.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:23 smithi177 ceph-mon[33043]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T21:00:24.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:23 smithi177 ceph-mon[33043]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T21:00:24.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:23 smithi177 ceph-mon[33043]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:00:24.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:23 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:24.724 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3[50762]: 2024-04-25T21:00:24.311+0000 7f827b91f700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T21:00:24.724 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3[50762]: 2024-04-25T21:00:24.317+0000 7f8272f09700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:00:24.805 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T21:00:24.806 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T21:00:24.806 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-25T21:00:24.806 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-25T21:00:24.806 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129305 s, 811 MB/s 2024-04-25T21:00:24.806 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-25T21:00:24.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-mon[29195]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:00:24.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-mon[29195]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T21:00:24.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:24.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:24.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:24 smithi037 ceph-mon[29195]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' 2024-04-25T21:00:25.119 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:24 smithi098 ceph-mon[32569]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:00:25.119 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:24 smithi098 ceph-mon[32569]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T21:00:25.120 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:24 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:25.120 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:24 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:25.120 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:24 smithi098 ceph-mon[32569]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' 2024-04-25T21:00:25.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:24 smithi177 ceph-mon[33043]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:00:25.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:24 smithi177 ceph-mon[33043]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T21:00:25.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:25.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:25.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:24 smithi177 ceph-mon[33043]: from='osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416]' entity='osd.3' 2024-04-25T21:00:25.455 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_3 2024-04-25T21:00:25.687 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] boot 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:25.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:25 smithi037 ceph-mon[29195]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] boot 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:26.059 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:25 smithi098 ceph-mon[32569]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: osd.3 [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] boot 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:00:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:25 smithi177 ceph-mon[33043]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2032889295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc103cb0 msgr2=0x7f0acc104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2032889295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc103cb0 0x7f0acc104110 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0abc0099c0 tx=0x7f0abc009cd0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2032889295 shutdown_connections 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2032889295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc104650 0x7f0acc10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2032889295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc102ab0 0x7f0acc102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2032889295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc103cb0 0x7f0acc104110 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2032889295 >> 172.21.15.98:0/2032889295 conn(0x7f0acc0fe010 msgr2=0x7f0acc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2032889295 shutdown_connections 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2032889295 wait complete. 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.334+0000 7f0ad35ee700 1 Processor -- start 2024-04-25T21:00:27.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 -- start start 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc103cb0 0x7f0acc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 0x7f0acc1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10e610 con 0x7f0acc103cb0 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10e490 con 0x7f0acc102ab0 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0acc10e790 con 0x7f0acc104650 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34858/0 (socket says 172.21.15.98:34858) 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 -- 172.21.15.98:0/2865279935 learned_addr learned my addr 172.21.15.98:0/2865279935 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad0b89700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc103cb0 0x7f0acc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad1b8b700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 0x7f0acc1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 -- 172.21.15.98:0/2865279935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 msgr2=0x7f0acc1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:27.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 0x7f0acc1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 -- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc103cb0 msgr2=0x7f0acc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc103cb0 0x7f0acc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 -- 172.21.15.98:0/2865279935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0acc1a0950 con 0x7f0acc102ab0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad1b8b700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 0x7f0acc1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad138a700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8007f10 tx=0x7f0ac800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac800ddd0 con 0x7f0acc102ab0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ac8018070 con 0x7f0acc102ab0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.335+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0acc1a0c40 con 0x7f0acc102ab0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.336+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac80125b0 con 0x7f0acc102ab0 2024-04-25T21:00:27.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.336+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0acc1a1130 con 0x7f0acc102ab0 2024-04-25T21:00:27.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.337+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc069730 con 0x7f0acc102ab0 2024-04-25T21:00:27.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.338+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ac80133f0 con 0x7f0acc102ab0 2024-04-25T21:00:27.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.339+0000 7f0ac27fc700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 0x7f0ab80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:27.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.339+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f0ac80cf350 con 0x7f0acc102ab0 2024-04-25T21:00:27.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.339+0000 7f0ad0b89700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 0x7f0ab80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:27.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.340+0000 7f0ad0b89700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 0x7f0ab80a6100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:27.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.343+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ac8010070 con 0x7f0acc102ab0 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi037 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.344 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:27.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi037 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:27.550+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0acc062e80 con 0x7f0ab80a3c40 2024-04-25T21:00:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T21:00:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi037 2024-04-25T21:00:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 8546M 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:28.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:28 smithi098 ceph-mon[32569]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T21:00:28.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:28 smithi098 ceph-mon[32569]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:28.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:28.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:28.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:28.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:28 smithi037 ceph-mon[29195]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T21:00:28.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:28 smithi037 ceph-mon[29195]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:28.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:28.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:28.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:28 smithi177 ceph-mon[33043]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T21:00:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:28 smithi177 ceph-mon[33043]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:29 smithi037 ceph-mon[29195]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:29 smithi037 ceph-mon[29195]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T21:00:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:29 smithi098 ceph-mon[32569]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:29.502 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:29 smithi098 ceph-mon[32569]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T21:00:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:29 smithi177 ceph-mon[33043]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 151 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:29 smithi177 ceph-mon[33043]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T21:00:31.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:31.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/676998961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]': finished 2024-04-25T21:00:31.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T21:00:31.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:31.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:31 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/1479232524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/676998961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]': finished 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:31 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/1479232524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/676998961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]: dispatch 2024-04-25T21:00:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "363aa3e4-6b35-48ce-a3a1-7e4b21c47651"}]': finished 2024-04-25T21:00:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T21:00:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:31 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/1479232524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:33.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:33 smithi037 ceph-mon[29195]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:00:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:33 smithi177 ceph-mon[33043]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:00:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:33 smithi098 ceph-mon[32569]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:00:35.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:35 smithi037 ceph-mon[29195]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:00:35.497 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:35 smithi098 ceph-mon[32569]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:00:35.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:35 smithi177 ceph-mon[33043]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:00:37.394 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:37 smithi098 ceph-mon[32569]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:00:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:37 smithi037 ceph-mon[29195]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:00:37.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:37 smithi177 ceph-mon[33043]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:00:38.476 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:38 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T21:00:38.476 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:38 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:38.476 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:38 smithi098 ceph-mon[32569]: Deploying daemon osd.4 on smithi098 2024-04-25T21:00:38.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:38 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T21:00:38.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:38 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:38.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:38 smithi037 ceph-mon[29195]: Deploying daemon osd.4 on smithi098 2024-04-25T21:00:38.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:38 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T21:00:38.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:38 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:38.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:38 smithi177 ceph-mon[33043]: Deploying daemon osd.4 on smithi098 2024-04-25T21:00:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:39 smithi037 ceph-mon[29195]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:39.539 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:39 smithi098 ceph-mon[32569]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:39 smithi177 ceph-mon[33043]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:41.411 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:41 smithi098 ceph-mon[32569]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:41.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:41 smithi037 ceph-mon[29195]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:41 smithi177 ceph-mon[33043]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:00:43.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:43 smithi037 ceph-mon[29195]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:43.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:43 smithi177 ceph-mon[33043]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:43 smithi098 ceph-mon[32569]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:44.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:45.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:45 smithi098 ceph-mon[32569]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:45 smithi037 ceph-mon[29195]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:45.741 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:45 smithi177 ceph-mon[33043]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:46.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.920+0000 7f0ac27fc700 1 -- 172.21.15.98:0/2865279935 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0acc062e80 con 0x7f0ab80a3c40 2024-04-25T21:00:46.921 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 4 on host 'smithi098' 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 msgr2=0x7f0ab80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 0x7f0ab80a6100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0abc006010 tx=0x7f0abc003860 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 msgr2=0x7f0acc198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8007f10 tx=0x7f0ac800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 shutdown_connections 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0acc104650 0x7f0acc1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ab80a3c40 0x7f0ab80a6100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0acc102ab0 0x7f0acc198d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 --2- 172.21.15.98:0/2865279935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0acc103cb0 0x7f0acc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 >> 172.21.15.98:0/2865279935 conn(0x7f0acc0fe010 msgr2=0x7f0acc1070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:46.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 shutdown_connections 2024-04-25T21:00:46.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:00:46.923+0000 7f0ad35ee700 1 -- 172.21.15.98:0/2865279935 wait complete. 2024-04-25T21:00:47.616 DEBUG:teuthology.orchestra.run.smithi098:osd.4> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T21:00:47.619 INFO:tasks.cephadm:Deploying osd.5 on smithi177 with /dev/vg_nvme/lv_3... 2024-04-25T21:00:47.619 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T21:00:47.862 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:48.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:47 smithi098 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:48.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:47 smithi037 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:48.248 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:47 smithi177 ceph-mon[33043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T21:00:49.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:49.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T21:00:49.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T21:00:49.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:49.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.418 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:00:49.220+0000 7f4a86b5d700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T21:00:49.418 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:00:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:00:49.225+0000 7f4a82950700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:49 smithi037 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T21:00:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T21:00:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: from='osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:49 smithi177 ceph-mon[33043]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111175 s, 943 MB/s 2024-04-25T21:00:49.791 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-04-25T21:00:50.408 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:50 smithi177 ceph-mon[33043]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:00:50.409 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:50 smithi177 ceph-mon[33043]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T21:00:50.409 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:50 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.409 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:50 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.438 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_3 2024-04-25T21:00:50.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:50 smithi037 ceph-mon[29195]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:00:50.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:50 smithi037 ceph-mon[29195]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T21:00:50.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:50 smithi098 ceph-mon[32569]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:00:50.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:50 smithi098 ceph-mon[32569]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T21:00:50.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:50 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:50 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:50.682 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:00:51.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:00:51.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:00:51.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:51.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] boot 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi098 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:00:51.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] boot 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi098 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:51.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: osd.4 [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] boot 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi098 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 8544M 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:51.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:00:52.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- 172.21.15.177:0/4064222967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 msgr2=0x7f05ec1040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.323 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 --2- 172.21.15.177:0/4064222967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec1040f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009990 tx=0x7f05d4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- 172.21.15.177:0/4064222967 shutdown_connections 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 --2- 172.21.15.177:0/4064222967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec104630 0x7f05ec10aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 --2- 172.21.15.177:0/4064222967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec102a90 0x7f05ec102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 --2- 172.21.15.177:0/4064222967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec1040f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- 172.21.15.177:0/4064222967 >> 172.21.15.177:0/4064222967 conn(0x7f05ec0fe030 msgr2=0x7f05ec100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- 172.21.15.177:0/4064222967 shutdown_connections 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- 172.21.15.177:0/4064222967 wait complete. 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 Processor -- start 2024-04-25T21:00:52.324 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.323+0000 7f05f21cd700 1 -- start start 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec102a90 0x7f05ec198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec104630 0x7f05ec1a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10e520 con 0x7f05ec102a90 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10e3a0 con 0x7f05ec104630 2024-04-25T21:00:52.326 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10e6a0 con 0x7f05ec103c90 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:38764/0 (socket says 172.21.15.177:38764) 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 -- 172.21.15.177:0/2449914538 learned_addr learned my addr 172.21.15.177:0/2449914538 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05eb7fe700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec102a90 0x7f05ec198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05ebfff700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec104630 0x7f05ec1a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 -- 172.21.15.177:0/2449914538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec104630 msgr2=0x7f05ec1a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec104630 0x7f05ec1a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 -- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec102a90 msgr2=0x7f05ec198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec102a90 0x7f05ec198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 -- 172.21.15.177:0/2449914538 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1a0880 con 0x7f05ec103c90 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e2ffd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4027070 con 0x7f05ec103c90 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec1a0b10 con 0x7f05ec103c90 2024-04-25T21:00:52.327 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05d4004000 con 0x7f05ec103c90 2024-04-25T21:00:52.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4022cf0 con 0x7f05ec103c90 2024-04-25T21:00:52.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.324+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a0fc0 con 0x7f05ec103c90 2024-04-25T21:00:52.328 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.325+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec069730 con 0x7f05ec103c90 2024-04-25T21:00:52.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.327+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f05d402a070 con 0x7f05ec103c90 2024-04-25T21:00:52.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.328+0000 7f05e97fa700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 0x7f05d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.328+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f05d40d03f0 con 0x7f05ec103c90 2024-04-25T21:00:52.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.328+0000 7f05eb7fe700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 0x7f05d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.329 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.328+0000 7f05eb7fe700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 0x7f05d80a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05ec19a110 tx=0x7f05dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.332 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.332+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05d4013070 con 0x7f05ec103c90 2024-04-25T21:00:52.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:52 smithi037 ceph-mon[29195]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T21:00:52.539 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:00:52.538+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f05ec062e80 con 0x7f05d80a3c40 2024-04-25T21:00:52.539 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:52 smithi177 ceph-mon[33043]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T21:00:52.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:52 smithi098 ceph-mon[32569]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T21:00:53.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:53 smithi177 ceph-mon[33043]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:53.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:53 smithi177 ceph-mon[33043]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:53.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:53.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:53.439 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:53 smithi098 ceph-mon[32569]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:53.695 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:53 smithi098 ceph-mon[32569]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:53.695 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:53.695 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:53.696 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:53 smithi037 ceph-mon[29195]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:53 smithi037 ceph-mon[29195]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:00:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:00:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:00:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/3034356756' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]': finished 2024-04-25T21:00:55.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T21:00:55.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:00:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:55.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/3034356756' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]': finished 2024-04-25T21:00:55.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T21:00:55.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/3034356756' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]: dispatch 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baed219b-bf19-48da-a89d-13bb3ff8485f"}]': finished 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T21:00:55.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:00:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:56 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/2054565178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:56.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:56 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/2054565178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:56.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:56 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/2054565178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:00:57.561 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:57 smithi098 ceph-mon[32569]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:57.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:57 smithi177 ceph-mon[33043]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:57.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:57 smithi037 ceph-mon[29195]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:59.500 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:00:59 smithi177 ceph-mon[33043]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:00:59 smithi098 ceph-mon[32569]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:00:59.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:00:59 smithi037 ceph-mon[29195]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:01.437 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:01 smithi177 ceph-mon[33043]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:01.555 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:01 smithi037 ceph-mon[29195]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:01.665 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:01 smithi098 ceph-mon[32569]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:03.467 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:03 smithi177 ceph-mon[33043]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:03.467 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T21:01:03.467 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:03.467 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:03 smithi177 ceph-mon[33043]: Deploying daemon osd.5 on smithi177 2024-04-25T21:01:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:03 smithi098 ceph-mon[32569]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:03.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T21:01:03.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:03.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:03 smithi098 ceph-mon[32569]: Deploying daemon osd.5 on smithi177 2024-04-25T21:01:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:03 smithi037 ceph-mon[29195]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T21:01:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:03 smithi037 ceph-mon[29195]: Deploying daemon osd.5 on smithi177 2024-04-25T21:01:05.485 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:05 smithi177 ceph-mon[33043]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:05 smithi098 ceph-mon[32569]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:05 smithi037 ceph-mon[29195]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:07.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:07 smithi177 ceph-mon[33043]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:07 smithi098 ceph-mon[32569]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:07 smithi037 ceph-mon[29195]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:09 smithi037 ceph-mon[29195]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:09 smithi177 ceph-mon[33043]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:09 smithi098 ceph-mon[32569]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:11.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:11 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:11.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:11 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:11 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:12.112 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.112+0000 7f05e97fa700 1 -- 172.21.15.177:0/2449914538 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05ec062e80 con 0x7f05d80a3c40 2024-04-25T21:01:12.113 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 5 on host 'smithi177' 2024-04-25T21:01:12.114 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 msgr2=0x7f05d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 0x7f05d80a6100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05ec19a110 tx=0x7f05dc00b410 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 msgr2=0x7f05ec199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006010 tx=0x7f05d40047b0 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 shutdown_connections 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05d80a3c40 0x7f05d80a6100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05ec104630 0x7f05ec1a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ec102a90 0x7f05ec198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 --2- 172.21.15.177:0/2449914538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05ec103c90 0x7f05ec199170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 >> 172.21.15.177:0/2449914538 conn(0x7f05ec0fe030 msgr2=0x7f05ec107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 shutdown_connections 2024-04-25T21:01:12.115 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:01:12.114+0000 7f05f21cd700 1 -- 172.21.15.177:0/2449914538 wait complete. 2024-04-25T21:01:12.777 DEBUG:teuthology.orchestra.run.smithi177:osd.5> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T21:01:12.780 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-25T21:01:12.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T21:01:13.036 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:13.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:13 smithi177 ceph-mon[33043]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.397 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:13 smithi037 ceph-mon[29195]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:13.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T21:01:13.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:13 smithi098 ceph-mon[32569]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-mon[33043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T21:01:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-mon[33043]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T21:01:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-mon[33043]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-mon[33043]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:14 smithi037 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T21:01:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:14 smithi037 ceph-mon[29195]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T21:01:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:14 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:14 smithi037 ceph-mon[29195]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:14 smithi037 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:14 smithi098 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T21:01:14.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:14 smithi098 ceph-mon[32569]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T21:01:14.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:14 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:14.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:14 smithi098 ceph-mon[32569]: from='osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:14 smithi098 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:01:14.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:01:14.525+0000 7feca4dd9700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T21:01:14.841 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:01:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:01:14.535+0000 7fec9c3c3700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:01:15.300 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T21:01:15.300 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T21:01:15.300 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T21:01:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T21:01:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110996 s, 945 MB/s 2024-04-25T21:01:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: from='osd.5 ' entity='osd.5' 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] boot 2024-04-25T21:01:15.301 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T21:01:15.302 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:15 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: from='osd.5 ' entity='osd.5' 2024-04-25T21:01:15.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] boot 2024-04-25T21:01:15.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T21:01:15.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:15 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: from='osd.5 ' entity='osd.5' 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: osd.5 [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] boot 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T21:01:15.343 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:15 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:01:15.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-25T21:01:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:01:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:01:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T21:01:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi177 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 8544M 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:16.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:16 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:01:16.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi177 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 8544M 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:16.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:16 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi177 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 8544M 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:16.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:16 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:17.290 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:17 smithi177 ceph-mon[33043]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:17.291 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:17 smithi177 ceph-mon[33043]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T21:01:17.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:17 smithi037 ceph-mon[29195]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:17.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:17 smithi037 ceph-mon[29195]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T21:01:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:17 smithi098 ceph-mon[32569]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:17 smithi098 ceph-mon[32569]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 -- 172.21.15.37:0/2205108303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 msgr2=0x7f5e68104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 --2- 172.21.15.37:0/2205108303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e68104120 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0099c0 tx=0x7f5e5c009cd0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 -- 172.21.15.37:0/2205108303 shutdown_connections 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 --2- 172.21.15.37:0/2205108303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 0x7f5e6810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 --2- 172.21.15.37:0/2205108303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 0x7f5e68102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 --2- 172.21.15.37:0/2205108303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e68104120 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 -- 172.21.15.37:0/2205108303 >> 172.21.15.37:0/2205108303 conn(0x7f5e680fe060 msgr2=0x7f5e681004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 -- 172.21.15.37:0/2205108303 shutdown_connections 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.914+0000 7f5e70456700 1 -- 172.21.15.37:0/2205108303 wait complete. 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 Processor -- start 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 -- start start 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 0x7f5e68198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 0x7f5e681a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810e530 con 0x7f5e68103cc0 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810e3b0 con 0x7f5e68104660 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e70456700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810e6b0 con 0x7f5e68102ac0 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e6d9f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e6d9f1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51044/0 (socket says 172.21.15.37:51044) 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e6d9f1700 1 -- 172.21.15.37:0/1066101053 learned_addr learned my addr 172.21.15.37:0/1066101053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.915+0000 7f5e6e1f2700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 0x7f5e68198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6e9f3700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 0x7f5e681a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 -- 172.21.15.37:0/1066101053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 msgr2=0x7f5e68198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 0x7f5e68198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 -- 172.21.15.37:0/1066101053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 msgr2=0x7f5e681a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 0x7f5e681a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 -- 172.21.15.37:0/1066101053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681a07e0 con 0x7f5e68103cc0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e6d9f1700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005910 tx=0x7f5e5c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c027070 con 0x7f5e68103cc0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e5c0041a0 con 0x7f5e68103cc0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c022da0 con 0x7f5e68103cc0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681a0a70 con 0x7f5e68103cc0 2024-04-25T21:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.916+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a0f50 con 0x7f5e68103cc0 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.917+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e68069730 con 0x7f5e68103cc0 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.918+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e5c02a070 con 0x7f5e68103cc0 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.919+0000 7f5e677fe700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 0x7f5e500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.919+0000 7f5e6e1f2700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 0x7f5e500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.919+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f5e5c0d0c90 con 0x7f5e68103cc0 2024-04-25T21:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.920+0000 7f5e6e1f2700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 0x7f5e500a6180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e68103b20 tx=0x7f5e600091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:17.923+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e5c013070 con 0x7f5e68103cc0 2024-04-25T21:01:18.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:18.145+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5e68062e80 con 0x7f5e500a3cc0 2024-04-25T21:01:19.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T21:01:19.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:19.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:19.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:19.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:19.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T21:01:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:19.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:19.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:19.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:20.326 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:20 smithi037 ceph-mon[29195]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T21:01:20.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:20 smithi098 ceph-mon[32569]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T21:01:20.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:20 smithi177 ceph-mon[33043]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T21:01:21.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:21 smithi098 ceph-mon[32569]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:21.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]: dispatch 2024-04-25T21:01:21.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]': finished 2024-04-25T21:01:21.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:21 smithi098 ceph-mon[32569]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T21:01:21.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:21 smithi037 ceph-mon[29195]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]: dispatch 2024-04-25T21:01:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]': finished 2024-04-25T21:01:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:21 smithi037 ceph-mon[29195]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T21:01:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:21 smithi177 ceph-mon[33043]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]: dispatch 2024-04-25T21:01:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/342593098' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54c67531-e8c5-4746-bfed-c8c47f4e63de"}]': finished 2024-04-25T21:01:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:21 smithi177 ceph-mon[33043]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T21:01:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:22.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:22 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4221137037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:22.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:22 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4221137037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:22.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:22 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4221137037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:23.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:23 smithi098 ceph-mon[32569]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:01:23.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:23 smithi037 ceph-mon[29195]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:01:23.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:23 smithi177 ceph-mon[33043]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:01:25.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:25 smithi037 ceph-mon[29195]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T21:01:25.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:25 smithi177 ceph-mon[33043]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T21:01:25.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:25 smithi098 ceph-mon[32569]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-25T21:01:27.333 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:27 smithi037 ceph-mon[29195]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:01:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:27 smithi177 ceph-mon[33043]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:01:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:27 smithi098 ceph-mon[32569]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:01:29.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:29 smithi037 ceph-mon[29195]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T21:01:29.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T21:01:29.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:29.479 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:29 smithi037 ceph-mon[29195]: Deploying daemon osd.6 on smithi037 2024-04-25T21:01:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:29 smithi177 ceph-mon[33043]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T21:01:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T21:01:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:29 smithi177 ceph-mon[33043]: Deploying daemon osd.6 on smithi037 2024-04-25T21:01:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:29 smithi098 ceph-mon[32569]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T21:01:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T21:01:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:29 smithi098 ceph-mon[32569]: Deploying daemon osd.6 on smithi037 2024-04-25T21:01:31.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:31 smithi037 ceph-mon[29195]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:01:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:31 smithi177 ceph-mon[33043]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:01:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:31 smithi098 ceph-mon[32569]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:01:33.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:33 smithi037 ceph-mon[29195]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:33 smithi177 ceph-mon[33043]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:33 smithi098 ceph-mon[32569]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:35 smithi177 ceph-mon[33043]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:35 smithi098 ceph-mon[32569]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:36.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:35 smithi037 ceph-mon[29195]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:36.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:36.225 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:38.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:37 smithi037 ceph-mon[29195]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:38.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:37 smithi177 ceph-mon[33043]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:37 smithi098 ceph-mon[32569]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:38.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.455+0000 7f5e677fe700 1 -- 172.21.15.37:0/1066101053 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e68062e80 con 0x7f5e500a3cc0 2024-04-25T21:01:38.456 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-04-25T21:01:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 msgr2=0x7f5e500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 0x7f5e500a6180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e68103b20 tx=0x7f5e600091c0 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 msgr2=0x7f5e681990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c005910 tx=0x7f5e5c009510 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 shutdown_connections 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5e500a3cc0 0x7f5e500a6180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e68104660 0x7f5e681a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e68102ac0 0x7f5e68198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 --2- 172.21.15.37:0/1066101053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e68103cc0 0x7f5e681990c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 >> 172.21.15.37:0/1066101053 conn(0x7f5e680fe060 msgr2=0x7f5e68107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 shutdown_connections 2024-04-25T21:01:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:01:38.458+0000 7f5e70456700 1 -- 172.21.15.37:0/1066101053 wait complete. 2024-04-25T21:01:39.047 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T21:01:39.050 INFO:tasks.cephadm:Deploying osd.7 on smithi098 with /dev/vg_nvme/lv_2... 2024-04-25T21:01:39.050 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T21:01:39.297 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:39 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.757 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:39 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:39 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-mon[29195]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T21:01:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-mon[29195]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T21:01:40.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-mon[29195]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:01:40.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:40.742 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:40 smithi098 ceph-mon[32569]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T21:01:40.742 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:40 smithi098 ceph-mon[32569]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T21:01:40.742 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:40 smithi098 ceph-mon[32569]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:01:40.743 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:40.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:40 smithi177 ceph-mon[33043]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T21:01:40.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:40 smithi177 ceph-mon[33043]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T21:01:40.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:40 smithi177 ceph-mon[33043]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:01:40.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.235 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:01:40.970+0000 7f5cdc5b0700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T21:01:41.235 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:01:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:01:40.979+0000 7f5cd3b9a700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:01:41.358 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T21:01:41.359 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T21:01:41.359 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-25T21:01:41.359 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-25T21:01:41.359 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128514 s, 816 MB/s 2024-04-25T21:01:41.359 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-25T21:01:41.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:41.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:01:41.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T21:01:41.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:41 smithi037 ceph-mon[29195]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.727 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:41 smithi098 ceph-mon[32569]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:41 smithi177 ceph-mon[33043]: from='osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250]' entity='osd.6' 2024-04-25T21:01:42.046 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_2 2024-04-25T21:01:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:01:42.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:01:42.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:01:42.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:42.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] boot 2024-04-25T21:01:42.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T21:01:42.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:42.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:01:42.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:01:42.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:42.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] boot 2024-04-25T21:01:42.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T21:01:42.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:42.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:01:42.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:01:42.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:42.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: osd.6 [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] boot 2024-04-25T21:01:42.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T21:01:42.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:01:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.025+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/625543848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4108780 msgr2=0x7fa4d410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.025+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/625543848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4108780 0x7fa4d410abf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00b0a0 tx=0x7fa4cc00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/625543848 shutdown_connections 2024-04-25T21:01:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/625543848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4108780 0x7fa4d410abf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/625543848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 0x7fa4d4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/625543848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4105e50 0x7fa4d4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/625543848 >> 172.21.15.98:0/625543848 conn(0x7fa4d40fcf30 msgr2=0x7fa4d40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/625543848 shutdown_connections 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/625543848 wait complete. 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 Processor -- start 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.026+0000 7fa4d9ff5700 1 -- start start 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 0x7fa4d4102e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4108780 0x7fa4d4101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410e540 con 0x7fa4d4108780 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410e3c0 con 0x7fa4d4105e50 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d410e6c0 con 0x7fa4d4103520 2024-04-25T21:01:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52600/0 (socket says 172.21.15.98:52600) 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 -- 172.21.15.98:0/4278413868 learned_addr learned my addr 172.21.15.98:0/4278413868 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d37fe700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 0x7fa4d4102e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d3fff700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4108780 0x7fa4d4101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 -- 172.21.15.98:0/4278413868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 msgr2=0x7fa4d4102e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 0x7fa4d4102e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 -- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4108780 msgr2=0x7fa4d4101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4108780 0x7fa4d4101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 -- 172.21.15.98:0/4278413868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d41022c0 con 0x7fa4d4105e50 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d2ffd700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400d950 tx=0x7fa4c400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4016070 con 0x7fa4d4105e50 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d41a2d40 con 0x7fa4d4105e50 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4c4009df0 con 0x7fa4d4105e50 2024-04-25T21:01:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400f3c0 con 0x7fa4d4105e50 2024-04-25T21:01:44.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.027+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d41a3220 con 0x7fa4d4105e50 2024-04-25T21:01:44.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.028+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d4069730 con 0x7fa4d4105e50 2024-04-25T21:01:44.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.030+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa4c4019070 con 0x7fa4d4105e50 2024-04-25T21:01:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.030+0000 7fa4d0ff9700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 0x7fa4bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.030+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fa4c40cf230 con 0x7fa4d4105e50 2024-04-25T21:01:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.030+0000 7fa4d37fe700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 0x7fa4bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.031+0000 7fa4d37fe700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 0x7fa4bc0a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8007c60 tx=0x7fa4c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.034+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4c409c920 con 0x7fa4d4105e50 2024-04-25T21:01:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi037 2024-04-25T21:01:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:44 smithi098 ceph-mon[32569]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T21:01:44.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:01:44.240+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa4d4062e80 con 0x7fa4bc0a3c40 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi037 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T21:01:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:44 smithi177 ceph-mon[33043]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi037 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 5697M 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:01:44.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:44 smithi037 ceph-mon[29195]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T21:01:45.117 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:45.117 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:45.117 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:45 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:45.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:45.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:45.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:45 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:45.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:01:45.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:01:45.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:45 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:46 smithi098 ceph-mon[32569]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:46 smithi098 ceph-mon[32569]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:46.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:46 smithi177 ceph-mon[33043]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:46.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:46 smithi177 ceph-mon[33043]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:46 smithi037 ceph-mon[29195]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:46.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:46 smithi037 ceph-mon[29195]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:01:48.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:48.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/45027788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]': finished 2024-04-25T21:01:48.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T21:01:48.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:01:48.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/1536867359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/45027788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]': finished 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:01:48.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/1536867359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:48.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:48.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/45027788' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]: dispatch 2024-04-25T21:01:48.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7826cf0-a44f-4917-bec2-bcb8aa242bd9"}]': finished 2024-04-25T21:01:48.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T21:01:48.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:01:48.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/1536867359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:01:50.287 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:50 smithi037 ceph-mon[29195]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:50.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:50 smithi177 ceph-mon[33043]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:50.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:50 smithi098 ceph-mon[32569]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:52.336 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:52 smithi098 ceph-mon[32569]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:52 smithi177 ceph-mon[33043]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:52.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:52 smithi037 ceph-mon[29195]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:54 smithi098 ceph-mon[32569]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:54.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:54 smithi177 ceph-mon[33043]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:54.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:54 smithi037 ceph-mon[29195]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:55.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T21:01:55.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:55.359 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T21:01:55.360 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:55.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T21:01:55.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:56.131 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:56 smithi098 ceph-mon[32569]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:56.131 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:56 smithi098 ceph-mon[32569]: Deploying daemon osd.7 on smithi098 2024-04-25T21:01:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:56 smithi177 ceph-mon[33043]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:56 smithi177 ceph-mon[33043]: Deploying daemon osd.7 on smithi098 2024-04-25T21:01:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:56 smithi037 ceph-mon[29195]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:56 smithi037 ceph-mon[29195]: Deploying daemon osd.7 on smithi098 2024-04-25T21:01:58.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:01:58 smithi177 ceph-mon[33043]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:58.397 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:01:58 smithi098 ceph-mon[32569]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:01:58.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:01:58 smithi037 ceph-mon[29195]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:00.145 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:00 smithi098 ceph-mon[32569]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:00.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:00 smithi177 ceph-mon[33043]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:00 smithi037 ceph-mon[29195]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:01 smithi177 ceph-mon[33043]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:01 smithi098 ceph-mon[32569]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:01 smithi037 ceph-mon[29195]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:03.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:03 smithi098 ceph-mon[32569]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:03.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:03 smithi177 ceph-mon[33043]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:03 smithi037 ceph-mon[29195]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:03.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.259+0000 7fa4d0ff9700 1 -- 172.21.15.98:0/4278413868 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4d4062e80 con 0x7fa4bc0a3c40 2024-04-25T21:02:04.260 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 7 on host 'smithi098' 2024-04-25T21:02:04.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 msgr2=0x7fa4bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:04.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 0x7fa4bc0a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8007c60 tx=0x7fa4c80073d0 comp rx=0 tx=0).stop 2024-04-25T21:02:04.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 msgr2=0x7fa4d41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400d950 tx=0x7fa4c400dc60 comp rx=0 tx=0).stop 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 shutdown_connections 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa4d4108780 0x7fa4d4101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4d4103520 0x7fa4d4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa4bc0a3c40 0x7fa4bc0a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 --2- 172.21.15.98:0/4278413868 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4d4105e50 0x7fa4d41014d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.261+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 >> 172.21.15.98:0/4278413868 conn(0x7fa4d40fcf30 msgr2=0x7fa4d4103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.262+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 shutdown_connections 2024-04-25T21:02:04.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:02:04.262+0000 7fa4d9ff5700 1 -- 172.21.15.98:0/4278413868 wait complete. 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.635 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:04.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi098:osd.7> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T21:02:04.992 INFO:tasks.cephadm:Deploying osd.8 on smithi177 with /dev/vg_nvme/lv_2... 2024-04-25T21:02:04.992 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T21:02:05.238 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:02:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T21:02:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T21:02:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:05.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:05 smithi177 ceph-mon[33043]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T21:02:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T21:02:05.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:05.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:05.668 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:02:05.321+0000 7f5233e58700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T21:02:05.668 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:02:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:02:05.329+0000 7f522fc4b700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: from='osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:05.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:05 smithi037 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] boot 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T21:02:06.531 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] boot 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T21:02:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: osd.7 [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] boot 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T21:02:06.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:02:07.318 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T21:02:07.318 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T21:02:07.319 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-04-25T21:02:07.319 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-04-25T21:02:07.319 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012351 s, 849 MB/s 2024-04-25T21:02:07.319 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-04-25T21:02:07.347 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:07 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:02:07.347 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:07 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:02:07.347 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:07 smithi177 ceph-mon[33043]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:07.347 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:07 smithi177 ceph-mon[33043]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T21:02:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:07 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:02:07.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:07 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:02:07.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:07 smithi098 ceph-mon[32569]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:07.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:07 smithi098 ceph-mon[32569]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T21:02:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:07 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:02:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:07 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:02:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:07 smithi037 ceph-mon[29195]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-04-25T21:02:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:07 smithi037 ceph-mon[29195]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T21:02:07.971 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_2 2024-04-25T21:02:08.222 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:02:09.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi098 2024-04-25T21:02:09.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:09.341 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:09 smithi177 ceph-mon[33043]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T21:02:09.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi098 2024-04-25T21:02:09.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:09.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:09 smithi098 ceph-mon[32569]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi098 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 5696M 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:09 smithi037 ceph-mon[29195]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T21:02:09.891 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.889+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/4130898119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c102ba0 msgr2=0x7fe39c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.889+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/4130898119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c102ba0 0x7fe39c103000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe390009990 tx=0x7fe390009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.889+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/4130898119 shutdown_connections 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.889+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/4130898119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c103540 0x7fe39c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.889+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/4130898119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c1019a0 0x7fe39c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/4130898119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c102ba0 0x7fe39c103000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/4130898119 >> 172.21.15.177:0/4130898119 conn(0x7fe39c0fcf40 msgr2=0x7fe39c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/4130898119 shutdown_connections 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/4130898119 wait complete. 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 Processor -- start 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- start start 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:09.892 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c102ba0 0x7fe39c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 0x7fe39c1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10d490 con 0x7fe39c103540 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10d310 con 0x7fe39c102ba0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.890+0000 7fe3a0ec5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c10d610 con 0x7fe39c1019a0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:43434/0 (socket says 172.21.15.177:43434) 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 -- 172.21.15.177:0/1825775921 learned_addr learned my addr 172.21.15.177:0/1825775921 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe399d9b700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c102ba0 0x7fe39c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 -- 172.21.15.177:0/1825775921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c102ba0 msgr2=0x7fe39c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39ad9d700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 0x7fe39c1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c102ba0 0x7fe39c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 -- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 msgr2=0x7fe39c1a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 0x7fe39c1a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 -- 172.21.15.177:0/1825775921 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a08e0 con 0x7fe39c1019a0 2024-04-25T21:02:09.893 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39ad9d700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 0x7fe39c1a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe39a59c700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00d980 tx=0x7fe38c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c017030 con 0x7fe39c1019a0 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a0bd0 con 0x7fe39c1019a0 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe38c0044d0 con 0x7fe39c1019a0 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c003aa0 con 0x7fe39c1019a0 2024-04-25T21:02:09.894 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.891+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a1160 con 0x7fe39c1019a0 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.892+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c069730 con 0x7fe39c1019a0 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.893+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe38c01a070 con 0x7fe39c1019a0 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.894+0000 7fe38b7fe700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.894+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7fe38c0cf5f0 con 0x7fe39c1019a0 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.894+0000 7fe399d9b700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:09.895 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.895+0000 7fe399d9b700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 0x7fe3840a6180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b5c0 tx=0x7fe390005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:09.898 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:09.898+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe38c09cbb0 con 0x7fe39c1019a0 2024-04-25T21:02:10.104 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:10.104+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe39c06ae60 con 0x7fe3840a3cc0 2024-04-25T21:02:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:10 smithi177 ceph-mon[33043]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T21:02:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:10 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:10 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:10.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:10 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:10.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:10 smithi098 ceph-mon[32569]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T21:02:10.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:10 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:10.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:10 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:10.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:10 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:10.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:10 smithi037 ceph-mon[29195]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T21:02:10.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:10 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:10.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:10 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:10.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:10 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:11 smithi177 ceph-mon[33043]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:11 smithi177 ceph-mon[33043]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:11 smithi177 ceph-mon[33043]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T21:02:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:11 smithi098 ceph-mon[32569]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:11 smithi098 ceph-mon[32569]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:11 smithi098 ceph-mon[32569]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T21:02:11.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:11 smithi037 ceph-mon[29195]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:11.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:11 smithi037 ceph-mon[29195]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:11.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:11 smithi037 ceph-mon[29195]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T21:02:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/2932086913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]': finished 2024-04-25T21:02:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T21:02:13.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:13.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:13.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/2932086913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]': finished 2024-04-25T21:02:13.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T21:02:13.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/2932086913' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]: dispatch 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2de6cccb-3606-4ff8-9aaa-fda637fade5f"}]': finished 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T21:02:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:14.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:14 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/3094659100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:14.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:14 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/3094659100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:14.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:14 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/3094659100' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:15.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:15 smithi177 ceph-mon[33043]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:15.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:15 smithi098 ceph-mon[32569]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:15.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:15 smithi037 ceph-mon[29195]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:17.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:17 smithi177 ceph-mon[33043]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:17 smithi098 ceph-mon[32569]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:17.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:17 smithi037 ceph-mon[29195]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:19.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:19 smithi098 ceph-mon[32569]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:19.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:19 smithi037 ceph-mon[29195]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:19.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:19 smithi177 ceph-mon[33043]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:20.636 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:20 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T21:02:20.636 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:20 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:20.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:20 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T21:02:20.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:20 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:20.831 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:20 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T21:02:20.831 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:20 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:21 smithi098 ceph-mon[32569]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:21 smithi098 ceph-mon[32569]: Deploying daemon osd.8 on smithi177 2024-04-25T21:02:21.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:21 smithi037 ceph-mon[29195]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:21.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:21 smithi037 ceph-mon[29195]: Deploying daemon osd.8 on smithi177 2024-04-25T21:02:21.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:21 smithi177 ceph-mon[33043]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:21.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:21 smithi177 ceph-mon[33043]: Deploying daemon osd.8 on smithi177 2024-04-25T21:02:23.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:23 smithi177 ceph-mon[33043]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:23.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:23 smithi098 ceph-mon[32569]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:23 smithi037 ceph-mon[29195]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:25.538 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:25 smithi177 ceph-mon[33043]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:25.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:25 smithi098 ceph-mon[32569]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:25 smithi037 ceph-mon[29195]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:27.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:26 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:26 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:26 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:27.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:26 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:26 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:26 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:27.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:26 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:26 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:27.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:26 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:27 smithi177 ceph-mon[33043]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:27 smithi098 ceph-mon[32569]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:28.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:27 smithi037 ceph-mon[29195]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:29.523 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.523+0000 7fe38b7fe700 1 -- 172.21.15.177:0/1825775921 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe39c06ae60 con 0x7fe3840a3cc0 2024-04-25T21:02:29.524 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 8 on host 'smithi177' 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 msgr2=0x7fe3840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 0x7fe3840a6180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b5c0 tx=0x7fe390005fd0 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 msgr2=0x7fe39c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00d980 tx=0x7fe38c00dd40 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 shutdown_connections 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39c103540 0x7fe39c1a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe39c1019a0 0x7fe39c198d00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 --2- 172.21.15.177:0/1825775921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe39c102ba0 0x7fe39c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:29.526 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.525+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 >> 172.21.15.177:0/1825775921 conn(0x7fe39c0fcf40 msgr2=0x7fe39c06a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.526+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 shutdown_connections 2024-04-25T21:02:29.527 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:02:29.526+0000 7fe3a0ec5700 1 -- 172.21.15.177:0/1825775921 wait complete. 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:29.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:29.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:29.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:30.257 DEBUG:teuthology.orchestra.run.smithi177:osd.8> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T21:02:30.260 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-25T21:02:30.260 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T21:02:30.518 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:30.817 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:30 smithi177 ceph-mon[33043]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:30.817 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:30 smithi177 ceph-mon[33043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:30.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:30 smithi098 ceph-mon[32569]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:30.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:30 smithi098 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:30.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:30 smithi037 ceph-mon[29195]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:30.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:30 smithi037 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:31.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:31 smithi177 ceph-mon[33043]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:31.981 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:31 smithi037 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:32.097 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:02:32.087+0000 7fe6fdd5b700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T21:02:32.097 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:02:32.096+0000 7fe6f5345700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:02:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T21:02:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: from='osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T21:02:32.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:32.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:31 smithi098 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112736 s, 930 MB/s 2024-04-25T21:02:32.739 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: from='osd.8 ' entity='osd.8' 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] boot 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T21:02:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:32 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: from='osd.8 ' entity='osd.8' 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] boot 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T21:02:33.036 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:32 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: from='osd.8 ' entity='osd.8' 2024-04-25T21:02:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: osd.8 [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] boot 2024-04-25T21:02:33.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: osdmap e59: 9 total, 9 up, 9 in 2024-04-25T21:02:33.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:32 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:02:33.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-25T21:02:33.692 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:33.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:33.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:33.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:33.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:33 smithi037 ceph-mon[29195]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:34.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:34.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:33 smithi177 ceph-mon[33043]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T21:02:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:02:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:02:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-25T21:02:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:34.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:33 smithi098 ceph-mon[32569]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T21:02:34.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:34 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi177 2024-04-25T21:02:34.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:34 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 5696M 2024-04-25T21:02:34.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:34 smithi037 ceph-mon[29195]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T21:02:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:34 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi177 2024-04-25T21:02:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:34 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 5696M 2024-04-25T21:02:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:34 smithi177 ceph-mon[33043]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T21:02:35.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:34 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi177 2024-04-25T21:02:35.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:34 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 5696M 2024-04-25T21:02:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:34 smithi098 ceph-mon[32569]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.404+0000 7f102c979700 1 -- 172.21.15.37:0/1073853134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028105ec0 msgr2=0x7f10281082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.404+0000 7f102c979700 1 --2- 172.21.15.37:0/1073853134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028105ec0 0x7f10281082b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f101c009990 tx=0x7f101c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 -- 172.21.15.37:0/1073853134 shutdown_connections 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 --2- 172.21.15.37:0/1073853134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10281087f0 0x7f102810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 --2- 172.21.15.37:0/1073853134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1028103590 0x7f1028105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 --2- 172.21.15.37:0/1073853134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028105ec0 0x7f10281082b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 -- 172.21.15.37:0/1073853134 >> 172.21.15.37:0/1073853134 conn(0x7f10280fcf60 msgr2=0x7f10280ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:35.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 -- 172.21.15.37:0/1073853134 shutdown_connections 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 -- 172.21.15.37:0/1073853134 wait complete. 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.405+0000 7f102c979700 1 Processor -- start 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 -- start start 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 0x7f1028101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10281087f0 0x7f1028101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f102810e530 con 0x7f1028103590 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f102810e3b0 con 0x7f1028105ec0 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102c979700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f102810e6b0 con 0x7f10281087f0 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38844/0 (socket says 172.21.15.37:38844) 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 -- 172.21.15.37:0/3758446368 learned_addr learned my addr 172.21.15.37:0/3758446368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f1025d9b700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 0x7f1028101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f1026d9d700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10281087f0 0x7f1028101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 -- 172.21.15.37:0/3758446368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10281087f0 msgr2=0x7f1028101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10281087f0 0x7f1028101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 -- 172.21.15.37:0/3758446368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 msgr2=0x7f1028101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 0x7f1028101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 -- 172.21.15.37:0/3758446368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1028102330 con 0x7f1028103590 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f1025d9b700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 0x7f1028101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:35.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.406+0000 7f102659c700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:35.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.407+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018016070 con 0x7f1028103590 2024-04-25T21:02:35.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.407+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281a2d90 con 0x7f1028103590 2024-04-25T21:02:35.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.407+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f101800fd40 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.407+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800e3b0 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.407+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281a3210 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.408+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1028069730 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.409+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1018019070 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.409+0000 7f10177fe700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 0x7f10100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.410+0000 7f1025d9b700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 0x7f10100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.410+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7f10180cf5a0 con 0x7f1028103590 2024-04-25T21:02:35.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.410+0000 7f1025d9b700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 0x7f10100a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f101c00b5c0 tx=0x7f101c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:35.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.414+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f101809c870 con 0x7f1028103590 2024-04-25T21:02:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:35.635+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1028062e80 con 0x7f10100a3c40 2024-04-25T21:02:35.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:35 smithi037 ceph-mon[29195]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:35.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:35.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:35.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:35 smithi177 ceph-mon[33043]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:35 smithi098 ceph-mon[32569]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:02:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:02:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:36.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:36 smithi037 ceph-mon[29195]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:36.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:36 smithi037 ceph-mon[29195]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T21:02:37.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:36 smithi177 ceph-mon[33043]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:37.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:36 smithi177 ceph-mon[33043]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T21:02:37.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:36 smithi098 ceph-mon[32569]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:02:37.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:36 smithi098 ceph-mon[32569]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T21:02:37.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:37 smithi037 ceph-mon[29195]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:37.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:37 smithi037 ceph-mon[29195]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T21:02:38.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:37 smithi177 ceph-mon[33043]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:38.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:37 smithi177 ceph-mon[33043]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T21:02:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:37 smithi098 ceph-mon[32569]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:37 smithi098 ceph-mon[32569]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T21:02:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:38 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]: dispatch 2024-04-25T21:02:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:38 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]': finished 2024-04-25T21:02:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:38 smithi037 ceph-mon[29195]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T21:02:38.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:38 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:39.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:38 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]: dispatch 2024-04-25T21:02:39.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:38 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]': finished 2024-04-25T21:02:39.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:38 smithi177 ceph-mon[33043]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T21:02:39.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:38 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:38 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]: dispatch 2024-04-25T21:02:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:38 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2100284362' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4fa00cfe-c3d2-4741-a451-b8a4c89b6753"}]': finished 2024-04-25T21:02:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:38 smithi098 ceph-mon[32569]: osdmap e64: 10 total, 9 up, 10 in 2024-04-25T21:02:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:38 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:39.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:39 smithi037 ceph-mon[29195]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T21:02:39.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:39 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2273335217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:40.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:39 smithi177 ceph-mon[33043]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T21:02:40.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:39 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2273335217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:39 smithi098 ceph-mon[32569]: pgmap v205: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-25T21:02:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:39 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2273335217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:02:41.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:41 smithi037 ceph-mon[29195]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T21:02:42.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:41 smithi177 ceph-mon[33043]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T21:02:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:41 smithi098 ceph-mon[32569]: pgmap v207: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-25T21:02:43.785 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:43 smithi037 ceph-mon[29195]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T21:02:44.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:43 smithi177 ceph-mon[33043]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T21:02:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:43 smithi098 ceph-mon[32569]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-25T21:02:45.968 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:45 smithi098 ceph-mon[32569]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:02:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:45 smithi037 ceph-mon[29195]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:02:46.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:45 smithi177 ceph-mon[33043]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:02:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T21:02:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:47.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T21:02:47.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:47.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T21:02:47.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:47 smithi037 ceph-mon[29195]: Deploying daemon osd.9 on smithi037 2024-04-25T21:02:47.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:47 smithi037 ceph-mon[29195]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:48.082 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:47 smithi177 ceph-mon[33043]: Deploying daemon osd.9 on smithi037 2024-04-25T21:02:48.082 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:47 smithi177 ceph-mon[33043]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:47 smithi098 ceph-mon[32569]: Deploying daemon osd.9 on smithi037 2024-04-25T21:02:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:47 smithi098 ceph-mon[32569]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:49 smithi037 ceph-mon[29195]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:50.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:49 smithi177 ceph-mon[33043]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:49 smithi098 ceph-mon[32569]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:51.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:51 smithi037 ceph-mon[29195]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:52.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:51 smithi177 ceph-mon[33043]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:51 smithi098 ceph-mon[32569]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:53 smithi037 ceph-mon[29195]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:53.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:53 smithi177 ceph-mon[33043]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:53.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:53 smithi098 ceph-mon[32569]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:53.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:55.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:55 smithi037 ceph-mon[29195]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:55.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:55 smithi177 ceph-mon[33043]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:55 smithi098 ceph-mon[32569]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:56.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.111+0000 7f10177fe700 1 -- 172.21.15.37:0/3758446368 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1028062e80 con 0x7f10100a3c40 2024-04-25T21:02:56.112 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-04-25T21:02:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.113+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 msgr2=0x7f10100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.113+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 0x7f10100a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f101c00b5c0 tx=0x7f101c005fb0 comp rx=0 tx=0).stop 2024-04-25T21:02:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.113+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 msgr2=0x7f1028102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:56.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.113+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f101800d950 tx=0x7f101800dc60 comp rx=0 tx=0).stop 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 shutdown_connections 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10281087f0 0x7f1028101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1028103590 0x7f1028102ef0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10100a3c40 0x7f10100a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 --2- 172.21.15.37:0/3758446368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1028105ec0 0x7f1028101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 >> 172.21.15.37:0/3758446368 conn(0x7f10280fcf60 msgr2=0x7f1028104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 shutdown_connections 2024-04-25T21:02:56.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:02:56.114+0000 7f102c979700 1 -- 172.21.15.37:0/3758446368 wait complete. 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:56.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:56 smithi037 ceph-mon[29195]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:56.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:56 smithi177 ceph-mon[33043]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T21:02:56.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:56.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:56 smithi098 ceph-mon[32569]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T21:02:56.895 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T21:02:56.898 INFO:tasks.cephadm:Deploying osd.10 on smithi098 with /dev/vg_nvme/lv_1... 2024-04-25T21:02:56.898 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T21:02:57.144 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:02:57.473 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:57 smithi037 ceph-mon[29195]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:57.473 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:57 smithi037 ceph-mon[29195]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T21:02:57.474 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:57 smithi037 ceph-mon[29195]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T21:02:57.474 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:57 smithi037 ceph-mon[29195]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:02:57.474 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:57.580 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:57 smithi098 ceph-mon[32569]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:57.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:57 smithi098 ceph-mon[32569]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T21:02:57.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:57 smithi098 ceph-mon[32569]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T21:02:57.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:57 smithi098 ceph-mon[32569]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:02:57.581 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:57 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:57 smithi177 ceph-mon[33043]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:57 smithi177 ceph-mon[33043]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T21:02:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:57 smithi177 ceph-mon[33043]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T21:02:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:57 smithi177 ceph-mon[33043]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-25T21:02:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:57 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:58.377 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:02:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:02:58.081+0000 7f1e1613a700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T21:02:58.378 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:02:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:02:58.095+0000 7f1e11f2d700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:02:59.318 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T21:02:59.318 INFO:teuthology.orchestra.run.smithi098.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T21:02:59.318 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10+0 records in 2024-04-25T21:02:59.319 INFO:teuthology.orchestra.run.smithi098.stderr:10+0 records out 2024-04-25T21:02:59.319 INFO:teuthology.orchestra.run.smithi098.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114898 s, 913 MB/s 2024-04-25T21:02:59.319 INFO:teuthology.orchestra.run.smithi098.stderr:--> Zapping successful for: 2024-04-25T21:02:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:59 smithi177 ceph-mon[33043]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:02:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:59 smithi177 ceph-mon[33043]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T21:02:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:59 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:02:59 smithi177 ceph-mon[33043]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:59.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:59 smithi098 ceph-mon[32569]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:02:59.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:59 smithi098 ceph-mon[32569]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T21:02:59.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:59 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.345 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:02:59 smithi098 ceph-mon[32569]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:02:59.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:59 smithi037 ceph-mon[29195]: from='osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-25T21:02:59.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:59 smithi037 ceph-mon[29195]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T21:02:59.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:59 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:02:59.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:02:59 smithi037 ceph-mon[29195]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T21:03:00.038 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi098:vg_nvme/lv_1 2024-04-25T21:03:00.283 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:03:00.309 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:00 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:03:00.310 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:00 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:03:00.310 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:00 smithi098 ceph-mon[32569]: osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] boot 2024-04-25T21:03:00.310 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:00 smithi098 ceph-mon[32569]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T21:03:00.310 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:00 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:03:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:00 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:03:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:00 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:03:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:00 smithi177 ceph-mon[33043]: osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] boot 2024-04-25T21:03:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:00 smithi177 ceph-mon[33043]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T21:03:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:00 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:03:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:00 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:03:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:00 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:03:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:00 smithi037 ceph-mon[29195]: osd.9 [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] boot 2024-04-25T21:03:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:00 smithi037 ceph-mon[29195]: osdmap e67: 10 total, 10 up, 10 in 2024-04-25T21:03:00.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:00 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:03:01.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T21:03:01.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:01.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi037 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:01.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:01.419 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T21:03:01.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi037 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:01.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T21:03:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi037 2024-04-25T21:03:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:01.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:02.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 -- 172.21.15.98:0/3033813687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 msgr2=0x7f4028101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:02.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 --2- 172.21.15.98:0/3033813687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f4028101dc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f401c0099c0 tx=0x7f401c009cd0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 -- 172.21.15.98:0/3033813687 shutdown_connections 2024-04-25T21:03:02.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 --2- 172.21.15.98:0/3033813687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028103540 0x7f4028109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 --2- 172.21.15.98:0/3033813687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f4028101dc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 --2- 172.21.15.98:0/3033813687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028102ba0 0x7f4028103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 -- 172.21.15.98:0/3033813687 >> 172.21.15.98:0/3033813687 conn(0x7f40280fcf40 msgr2=0x7f40280ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 -- 172.21.15.98:0/3033813687 shutdown_connections 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.150+0000 7f402d85a700 1 -- 172.21.15.98:0/3033813687 wait complete. 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 Processor -- start 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 -- start start 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028102ba0 0x7f402819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028103540 0x7f40281a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f402810d4e0 con 0x7f4028103540 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f402810d360 con 0x7f40281019a0 2024-04-25T21:03:02.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f402d85a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f402810d660 con 0x7f4028102ba0 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f4026ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f4026ffd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33534/0 (socket says 172.21.15.98:33534) 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f4026ffd700 1 -- 172.21.15.98:0/2262759193 learned_addr learned my addr 172.21.15.98:0/2262759193 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f40267fc700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028102ba0 0x7f402819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.151+0000 7f40277fe700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028103540 0x7f40281a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 -- 172.21.15.98:0/2262759193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028102ba0 msgr2=0x7f402819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028102ba0 0x7f402819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 -- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028103540 msgr2=0x7f40281a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028103540 0x7f40281a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 -- 172.21.15.98:0/2262759193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40281a0950 con 0x7f40281019a0 2024-04-25T21:03:02.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f4026ffd700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f401c006010 tx=0x7f401c005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c026070 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f401c004020 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40281a0be0 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c00fe20 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.152+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40281a1030 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.153+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402802df80 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.154+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f401c029070 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.155+0000 7f402c858700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.155+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f401c0d0670 con 0x7f40281019a0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.155+0000 7f40267fc700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:02.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.155+0000 7f40267fc700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 0x7f40100a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f402806b800 tx=0x7f4018009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:02.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.160+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f401c05dff0 con 0x7f40281019a0 2024-04-25T21:03:02.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:02.363+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4028062e80 con 0x7f40100a3c40 2024-04-25T21:03:03.227 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:03 smithi098 ceph-mon[32569]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:03.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:03 smithi098 ceph-mon[32569]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:03.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:03.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:03.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:03.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:03 smithi037 ceph-mon[29195]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:03.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:03 smithi037 ceph-mon[29195]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:03.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:03.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:03.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:03 smithi177 ceph-mon[33043]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:03 smithi177 ceph-mon[33043]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi098:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:05.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:05 smithi037 ceph-mon[29195]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:05 smithi177 ceph-mon[33043]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:05.614 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:05 smithi098 ceph-mon[32569]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/2915915159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]': finished 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:06.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:06 smithi037 ceph-mon[29195]: from='client.? 172.21.15.98:0/3040747142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/2915915159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]': finished 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:06 smithi177 ceph-mon[33043]: from='client.? 172.21.15.98:0/3040747142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:06.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/2915915159' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]: dispatch 2024-04-25T21:03:06.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3397e1ac-4efe-4cd8-8a27-a2f07662d081"}]': finished 2024-04-25T21:03:06.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: osdmap e69: 11 total, 10 up, 11 in 2024-04-25T21:03:06.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:06.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:06 smithi098 ceph-mon[32569]: from='client.? 172.21.15.98:0/3040747142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:07.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:07 smithi098 ceph-mon[32569]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:07.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:07 smithi177 ceph-mon[33043]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:07.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:07 smithi037 ceph-mon[29195]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:09 smithi177 ceph-mon[33043]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:09.660 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:09 smithi098 ceph-mon[32569]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:09.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:09 smithi037 ceph-mon[29195]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:11.589 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:11 smithi098 ceph-mon[32569]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:11 smithi177 ceph-mon[33043]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:11.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:11 smithi037 ceph-mon[29195]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:13.526 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:13 smithi098 ceph-mon[32569]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:13.527 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T21:03:13.527 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:13 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:13.527 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:13 smithi098 ceph-mon[32569]: Deploying daemon osd.10 on smithi098 2024-04-25T21:03:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:13 smithi177 ceph-mon[33043]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T21:03:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:13 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:13.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:13 smithi177 ceph-mon[33043]: Deploying daemon osd.10 on smithi098 2024-04-25T21:03:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:13 smithi037 ceph-mon[29195]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T21:03:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:13 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:13.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:13 smithi037 ceph-mon[29195]: Deploying daemon osd.10 on smithi098 2024-04-25T21:03:15.332 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:15 smithi098 ceph-mon[32569]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:15.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:15 smithi177 ceph-mon[33043]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:15.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:15 smithi037 ceph-mon[29195]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:17.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:17 smithi098 ceph-mon[32569]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:17.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:17 smithi177 ceph-mon[33043]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:17.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:17 smithi037 ceph-mon[29195]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:19 smithi098 ceph-mon[32569]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:19 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:20.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:19 smithi037 ceph-mon[29195]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:20.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:19 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:20.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:19 smithi177 ceph-mon[33043]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:20.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:20.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:19 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:21 smithi098 ceph-mon[32569]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:22.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:21 smithi037 ceph-mon[29195]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:22.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:21 smithi177 ceph-mon[33043]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:22.842 INFO:teuthology.orchestra.run.smithi098.stdout:Created osd(s) 10 on host 'smithi098' 2024-04-25T21:03:22.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.841+0000 7f402c858700 1 -- 172.21.15.98:0/2262759193 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4028062e80 con 0x7f40100a3c40 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 msgr2=0x7f40100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 0x7f40100a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f402806b800 tx=0x7f4018009190 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 msgr2=0x7f402806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f401c006010 tx=0x7f401c005af0 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 shutdown_connections 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4028103540 0x7f40281a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f40100a3c40 0x7f40100a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40281019a0 0x7f402806b280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 --2- 172.21.15.98:0/2262759193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4028102ba0 0x7f402819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 >> 172.21.15.98:0/2262759193 conn(0x7f40280fcf40 msgr2=0x7f4028105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 shutdown_connections 2024-04-25T21:03:22.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-04-25T21:03:22.844+0000 7f402d85a700 1 -- 172.21.15.98:0/2262759193 wait complete. 2024-04-25T21:03:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:23.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:23.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:23.670 DEBUG:teuthology.orchestra.run.smithi098:osd.10> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T21:03:23.673 INFO:tasks.cephadm:Deploying osd.11 on smithi177 with /dev/vg_nvme/lv_1... 2024-04-25T21:03:23.673 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T21:03:23.919 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:24.042 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:23 smithi177 ceph-mon[33043]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:24.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T21:03:24.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T21:03:24.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:24.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:24.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:24.168 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:03:23.943+0000 7f34c1f8f700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T21:03:24.168 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:03:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:03:23.954+0000 7f34bdd82700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: from='osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:23 smithi037 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]: dispatch 2024-04-25T21:03:25.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:24 smithi098 ceph-mon[32569]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:03:25.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:24 smithi098 ceph-mon[32569]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T21:03:25.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:24 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:25.228 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:24 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:25.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:24 smithi037 ceph-mon[29195]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:03:25.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:24 smithi037 ceph-mon[29195]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T21:03:25.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:25.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:24 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:24 smithi177 ceph-mon[33043]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi098", "root=default"]}]': finished 2024-04-25T21:03:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:24 smithi177 ceph-mon[33043]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T21:03:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:24 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114463 s, 916 MB/s 2024-04-25T21:03:26.081 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] boot 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:25 smithi037 ceph-mon[29195]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] boot 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:25 smithi177 ceph-mon[33043]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: osd.10 [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] boot 2024-04-25T21:03:26.372 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: osdmap e72: 11 total, 11 up, 11 in 2024-04-25T21:03:26.373 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:03:26.373 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:25 smithi098 ceph-mon[32569]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T21:03:26.890 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi177:vg_nvme/lv_1 2024-04-25T21:03:27.137 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi098 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:27.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi098 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:27.544 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi098 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 4272M 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:27.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:28.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:28 smithi037 ceph-mon[29195]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T21:03:28.559 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:28 smithi177 ceph-mon[33043]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T21:03:28.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:28 smithi098 ceph-mon[32569]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 -- 172.21.15.177:0/3434414257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0102aa0 msgr2=0x7f55a0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 --2- 172.21.15.177:0/3434414257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0102aa0 0x7f55a0102ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f559c009990 tx=0x7f559c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 -- 172.21.15.177:0/3434414257 shutdown_connections 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 --2- 172.21.15.177:0/3434414257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0104640 0x7f55a010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 --2- 172.21.15.177:0/3434414257 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0102aa0 0x7f55a0102ec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.940+0000 7f55a87df700 1 --2- 172.21.15.177:0/3434414257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0103ca0 0x7f55a0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.960 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 -- 172.21.15.177:0/3434414257 >> 172.21.15.177:0/3434414257 conn(0x7f55a00fe040 msgr2=0x7f55a0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 -- 172.21.15.177:0/3434414257 shutdown_connections 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 -- 172.21.15.177:0/3434414257 wait complete. 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 Processor -- start 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 -- start start 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.941+0000 7f55a87df700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0102aa0 0x7f55a0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0104640 0x7f55a01a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010e530 con 0x7f55a0102aa0 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010e3b0 con 0x7f55a0104640 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55a010e6b0 con 0x7f55a0103ca0 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.961 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:40760/0 (socket says 172.21.15.177:40760) 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 -- 172.21.15.177:0/1980975057 learned_addr learned my addr 172.21.15.177:0/1980975057 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a657b700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0102aa0 0x7f55a0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 -- 172.21.15.177:0/1980975057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0104640 msgr2=0x7f55a01a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0104640 0x7f55a01a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 -- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0102aa0 msgr2=0x7f55a0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0102aa0 0x7f55a0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 -- 172.21.15.177:0/1980975057 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a01a08a0 con 0x7f55a0103ca0 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a5d7a700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f559000c970 tx=0x7f559000cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590007aa0 con 0x7f55a0103ca0 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a01a0b90 con 0x7f55a0103ca0 2024-04-25T21:03:28.962 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5590004ce0 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900055c0 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.942+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a01a1080 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.943+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a0069730 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.944+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5590007c40 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.945+0000 7f55977fe700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 0x7f558c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.945+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f55900cfc10 con 0x7f55a0103ca0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.945+0000 7f55a657b700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 0x7f558c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.945+0000 7f55a657b700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 0x7f558c0a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f559c003790 tx=0x7f559c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:28.963 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:28.949+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f559009ccd0 con 0x7f55a0103ca0 2024-04-25T21:03:29.157 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:29.156+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f55a0062e80 con 0x7f558c0a3c40 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: pgmap v242: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:29.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:29 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: pgmap v242: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T21:03:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T21:03:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:29 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: pgmap v242: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T21:03:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T21:03:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T21:03:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T21:03:29.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:29 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:30 smithi177 ceph-mon[33043]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:30.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:30 smithi098 ceph-mon[32569]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:30.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:30 smithi037 ceph-mon[29195]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:03:31.572 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:31 smithi037 ceph-mon[29195]: pgmap v244: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:31 smithi177 ceph-mon[33043]: pgmap v244: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:31 smithi098 ceph-mon[32569]: pgmap v244: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T21:03:32.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:32 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/1739133111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:32 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:32 smithi177 ceph-mon[33043]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]': finished 2024-04-25T21:03:32.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:32 smithi177 ceph-mon[33043]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T21:03:32.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:32 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:32 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/1739133111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:32 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:32 smithi098 ceph-mon[32569]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]': finished 2024-04-25T21:03:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:32 smithi098 ceph-mon[32569]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T21:03:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:32 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:32.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:32 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/1739133111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:32 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]: dispatch 2024-04-25T21:03:32.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:32 smithi037 ceph-mon[29195]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52308cf8-3492-4a38-b722-87b7bf3fb5c0"}]': finished 2024-04-25T21:03:32.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:32 smithi037 ceph-mon[29195]: osdmap e77: 12 total, 11 up, 12 in 2024-04-25T21:03:32.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:32 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:33 smithi177 ceph-mon[33043]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:03:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:33 smithi177 ceph-mon[33043]: from='client.? 172.21.15.177:0/2250631243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:33 smithi177 ceph-mon[33043]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T21:03:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:33 smithi177 ceph-mon[33043]: Cluster is now healthy 2024-04-25T21:03:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:33 smithi098 ceph-mon[32569]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:03:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:33 smithi098 ceph-mon[32569]: from='client.? 172.21.15.177:0/2250631243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:33 smithi098 ceph-mon[32569]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T21:03:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:33 smithi098 ceph-mon[32569]: Cluster is now healthy 2024-04-25T21:03:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:33 smithi037 ceph-mon[29195]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-25T21:03:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:33 smithi037 ceph-mon[29195]: from='client.? 172.21.15.177:0/2250631243' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T21:03:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:33 smithi037 ceph-mon[29195]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T21:03:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:33 smithi037 ceph-mon[29195]: Cluster is now healthy 2024-04-25T21:03:35.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:35 smithi177 ceph-mon[33043]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:03:35.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:35 smithi098 ceph-mon[32569]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:03:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:35 smithi037 ceph-mon[29195]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T21:03:37.419 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:37 smithi177 ceph-mon[33043]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:03:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:37 smithi098 ceph-mon[32569]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:03:37.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:37 smithi037 ceph-mon[29195]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T21:03:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:39 smithi177 ceph-mon[33043]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:39.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:39 smithi098 ceph-mon[32569]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:39.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:39 smithi037 ceph-mon[29195]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:40.545 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T21:03:40.545 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:40 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:40.545 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:40 smithi177 ceph-mon[33043]: Deploying daemon osd.11 on smithi177 2024-04-25T21:03:40.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T21:03:40.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:40 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:40.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:40 smithi098 ceph-mon[32569]: Deploying daemon osd.11 on smithi177 2024-04-25T21:03:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T21:03:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:40 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:40.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:40 smithi037 ceph-mon[29195]: Deploying daemon osd.11 on smithi177 2024-04-25T21:03:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:41 smithi177 ceph-mon[33043]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:41 smithi098 ceph-mon[32569]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:41 smithi037 ceph-mon[29195]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T21:03:43.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:43 smithi177 ceph-mon[33043]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:43 smithi098 ceph-mon[32569]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:43.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:43 smithi037 ceph-mon[29195]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:45.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:45 smithi177 ceph-mon[33043]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:45.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:45 smithi098 ceph-mon[32569]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:45 smithi037 ceph-mon[29195]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:46.952 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:46.952 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:46.952 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:46 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:46.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:46.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:46.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:46 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:47.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:47.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:47.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:46 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:47.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:47 smithi037 ceph-mon[29195]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:48.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:47 smithi177 ceph-mon[33043]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:47 smithi098 ceph-mon[32569]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:49.628 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.628+0000 7f55977fe700 1 -- 172.21.15.177:0/1980975057 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55a0062e80 con 0x7f558c0a3c40 2024-04-25T21:03:49.628 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 11 on host 'smithi177' 2024-04-25T21:03:49.630 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 msgr2=0x7f558c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 0x7f558c0a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f559c003790 tx=0x7f559c005fb0 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 msgr2=0x7f55a0199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f559000c970 tx=0x7f559000cd30 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 shutdown_connections 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f558c0a3c40 0x7f558c0a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55a0104640 0x7f55a01a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55a0102aa0 0x7f55a0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 --2- 172.21.15.177:0/1980975057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55a0103ca0 0x7f55a0199190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 >> 172.21.15.177:0/1980975057 conn(0x7f55a00fe040 msgr2=0x7f55a0107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 shutdown_connections 2024-04-25T21:03:49.631 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:03:49.630+0000 7f55a87df700 1 -- 172.21.15.177:0/1980975057 wait complete. 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:49.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:49.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:49 smithi177 ceph-mon[33043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:50.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:50.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:49 smithi098 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:50.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:49 smithi037 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T21:03:50.331 DEBUG:teuthology.orchestra.run.smithi177:osd.11> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T21:03:50.334 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T21:03:50.334 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T21:03:50.594 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:03:51.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:51.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T21:03:51.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T21:03:51.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:51.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:51.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:51 smithi037 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:51.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:51 smithi177 ceph-mon[33043]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T21:03:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T21:03:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: from='osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:51.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:51 smithi098 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2024-04-25T21:03:52.221 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:03:52.102+0000 7fc35eead700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T21:03:52.221 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:03:52.115+0000 7fc356c98700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T21:03:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 -- 172.21.15.37:0/509492158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc103ca0 msgr2=0x7f43fc104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:52.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 --2- 172.21.15.37:0/509492158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc103ca0 0x7f43fc104100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43e80099c0 tx=0x7f43e8009cd0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 -- 172.21.15.37:0/509492158 shutdown_connections 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 --2- 172.21.15.37:0/509492158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 --2- 172.21.15.37:0/509492158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc102aa0 0x7f43fc102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 --2- 172.21.15.37:0/509492158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc103ca0 0x7f43fc104100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 -- 172.21.15.37:0/509492158 >> 172.21.15.37:0/509492158 conn(0x7f43fc0fe040 msgr2=0x7f43fc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.326+0000 7f4402f8c700 1 -- 172.21.15.37:0/509492158 shutdown_connections 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.327+0000 7f4402f8c700 1 -- 172.21.15.37:0/509492158 wait complete. 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 Processor -- start 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 -- start start 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc103ca0 0x7f43fc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e560 con 0x7f43fc102aa0 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e3e0 con 0x7f43fc103ca0 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4402f8c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f43fc10e6e0 con 0x7f43fc104640 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36876/0 (socket says 172.21.15.37:36876) 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 -- 172.21.15.37:0/1862626272 learned_addr learned my addr 172.21.15.37:0/1862626272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4401529700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f43fbfff700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc103ca0 0x7f43fc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 -- 172.21.15.37:0/1862626272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 msgr2=0x7f43fc1a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc1a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 -- 172.21.15.37:0/1862626272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc103ca0 msgr2=0x7f43fc199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:52.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc103ca0 0x7f43fc199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 -- 172.21.15.37:0/1862626272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43fc1a0780 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4401529700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc1a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.328+0000 7f4400d28700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c9c0 tx=0x7f43f000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.329+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0007ac0 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.329+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43fc1a0a70 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.329+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f0007c60 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.329+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0018660 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.329+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43fc1a0fb0 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.330+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43fc069730 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.331+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43f000f450 con 0x7f43fc102aa0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.332+0000 7f43f9ffb700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 0x7f43ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.332+0000 7f43fbfff700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 0x7f43ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.332+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f43f00cfe90 con 0x7f43fc102aa0 2024-04-25T21:03:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.332+0000 7f43fbfff700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 0x7f43ec0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:52.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.337+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43f0014030 con 0x7f43fc102aa0 2024-04-25T21:03:52.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.546+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f43fc063650 con 0x7f43fc102aa0 2024-04-25T21:03:52.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.546+0000 7f43f9ffb700 1 -- 172.21.15.37:0/1862626272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f43f009ce30 con 0x7f43fc102aa0 2024-04-25T21:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:03:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 msgr2=0x7f43ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 0x7f43ec0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43e8000c00 tx=0x7f43e8005fb0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 msgr2=0x7f43fc198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c9c0 tx=0x7f43f000cd80 comp rx=0 tx=0).stop 2024-04-25T21:03:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 shutdown_connections 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43fc104640 0x7f43fc1a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43fc102aa0 0x7f43fc198b20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f43ec0a3c40 0x7f43ec0a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 --2- 172.21.15.37:0/1862626272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43fc103ca0 0x7f43fc199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 >> 172.21.15.37:0/1862626272 conn(0x7f43fc0fe040 msgr2=0x7f43fc106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 shutdown_connections 2024-04-25T21:03:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:52.549+0000 7f4402f8c700 1 -- 172.21.15.37:0/1862626272 wait complete. 2024-04-25T21:03:52.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:03:52.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T21:03:52.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:52.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:52.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: from='osd.11 ' entity='osd.11' 2024-04-25T21:03:52.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:52 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1862626272' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:52.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:03:52.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T21:03:52.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:52.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:52.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: from='osd.11 ' entity='osd.11' 2024-04-25T21:03:52.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:52 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1862626272' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: from='osd.11 ' entity='osd.11' 2024-04-25T21:03:53.064 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:52 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1862626272' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:53.191 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1714079004,"num_in_osds":12,"osd_in_since":1714079012,"num_remapped_pgs":0} 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: purged_snaps scrub starts 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: purged_snaps scrub ok 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] boot 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T21:03:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:53.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: purged_snaps scrub starts 2024-04-25T21:03:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: purged_snaps scrub ok 2024-04-25T21:03:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] boot 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:53.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: purged_snaps scrub starts 2024-04-25T21:03:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: purged_snaps scrub ok 2024-04-25T21:03:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-04-25T21:03:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: osd.11 [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] boot 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:53.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:53.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:03:54.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T21:03:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:03:54.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:54 smithi037 ceph-mon[29195]: Detected new or changed devices on smithi177 2024-04-25T21:03:54.908 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:54 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 4272M 2024-04-25T21:03:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:54 smithi098 ceph-mon[32569]: Detected new or changed devices on smithi177 2024-04-25T21:03:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:54 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 4272M 2024-04-25T21:03:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:54 smithi177 ceph-mon[33043]: Detected new or changed devices on smithi177 2024-04-25T21:03:55.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:54 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 4272M 2024-04-25T21:03:55.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:55 smithi037 ceph-mon[29195]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:55.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:55 smithi037 ceph-mon[29195]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T21:03:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:55 smithi098 ceph-mon[32569]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:55 smithi098 ceph-mon[32569]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T21:03:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:55 smithi177 ceph-mon[33043]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:55 smithi177 ceph-mon[33043]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 -- 172.21.15.37:0/1135088770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 msgr2=0x7f11c4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/1135088770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c4103000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f11b80099c0 tx=0x7f11b8009cd0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 -- 172.21.15.37:0/1135088770 shutdown_connections 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/1135088770 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c4103540 0x7f11c4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/1135088770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c41019a0 0x7f11c4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/1135088770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c4103000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 -- 172.21.15.37:0/1135088770 >> 172.21.15.37:0/1135088770 conn(0x7f11c40fcf40 msgr2=0x7f11c40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:56.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 -- 172.21.15.37:0/1135088770 shutdown_connections 2024-04-25T21:03:56.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.159+0000 7f11c8bf9700 1 -- 172.21.15.37:0/1135088770 wait complete. 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 Processor -- start 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 -- start start 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 0x7f11c406b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c4103540 0x7f11c419ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.160+0000 7f11c8bf9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11c410d430 con 0x7f11c4102ba0 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c8bf9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f11c410d2b0 con 0x7f11c41019a0 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c8bf9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f11c410d5b0 con 0x7f11c4103540 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36896/0 (socket says 172.21.15.37:36896) 2024-04-25T21:03:56.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 -- 172.21.15.37:0/3240548541 learned_addr learned my addr 172.21.15.37:0/3240548541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c259c700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 0x7f11c406b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c2d9d700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c4103540 0x7f11c419ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 -- 172.21.15.37:0/3240548541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c4103540 msgr2=0x7f11c419ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c4103540 0x7f11c419ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 -- 172.21.15.37:0/3240548541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 msgr2=0x7f11c406b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 0x7f11c406b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 -- 172.21.15.37:0/3240548541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c41a0840 con 0x7f11c4102ba0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c259c700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 0x7f11c406b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c1d9b700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b580 tx=0x7f11b8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8027070 con 0x7f11c4102ba0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b8005960 con 0x7f11c4102ba0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c41a0ad0 con 0x7f11c4102ba0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.161+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8022c10 con 0x7f11c4102ba0 2024-04-25T21:03:56.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.162+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41a1000 con 0x7f11c4102ba0 2024-04-25T21:03:56.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.163+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c4069730 con 0x7f11c4102ba0 2024-04-25T21:03:56.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.164+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f11b802a070 con 0x7f11c4102ba0 2024-04-25T21:03:56.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.164+0000 7f11b37fe700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 0x7f11ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:56.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.164+0000 7f11c259c700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 0x7f11ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:56.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.165+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f11b80d0f50 con 0x7f11c4102ba0 2024-04-25T21:03:56.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.165+0000 7f11c259c700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 0x7f11ac0a6180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005950 tx=0x7f11b400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:56.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.168+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11b8013070 con 0x7f11c4102ba0 2024-04-25T21:03:56.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.377+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f11c402e190 con 0x7f11c4102ba0 2024-04-25T21:03:56.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.378+0000 7f11b37fe700 1 -- 172.21.15.37:0/3240548541 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f11b809ddc0 con 0x7f11c4102ba0 2024-04-25T21:03:56.378 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:03:56.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 msgr2=0x7f11ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 0x7f11ac0a6180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005950 tx=0x7f11b400a300 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 msgr2=0x7f11c419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b580 tx=0x7f11b8009580 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 shutdown_connections 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11c4103540 0x7f11c419ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f11ac0a3cc0 0x7f11ac0a6180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c41019a0 0x7f11c406b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 --2- 172.21.15.37:0/3240548541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11c4102ba0 0x7f11c419d920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 >> 172.21.15.37:0/3240548541 conn(0x7f11c40fcf40 msgr2=0x7f11c4105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 shutdown_connections 2024-04-25T21:03:56.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:56.380+0000 7f11c8bf9700 1 -- 172.21.15.37:0/3240548541 wait complete. 2024-04-25T21:03:56.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:56 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3240548541' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:56.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:56 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3240548541' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:57.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:56 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3240548541' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T21:03:57.189 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714079033,"num_in_osds":12,"osd_in_since":1714079012,"num_remapped_pgs":0} 2024-04-25T21:03:57.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:03:57.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:03:57.814 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:57 smithi037 ceph-mon[29195]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:57.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:57 smithi098 ceph-mon[32569]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:57 smithi177 ceph-mon[33043]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1349987375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14108a80 msgr2=0x7f9b14108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1349987375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14108a80 0x7f9b14108e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009990 tx=0x7f9b04009ca0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1349987375 shutdown_connections 2024-04-25T21:03:59.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1349987375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14103450 0x7f9b1410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1349987375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14108a80 0x7f9b14108e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1349987375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14102a90 0x7f9b14102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1349987375 >> 172.21.15.37:0/1349987375 conn(0x7f9b14079bf0 msgr2=0x7f9b1407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1349987375 shutdown_connections 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.186+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1349987375 wait complete. 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 Processor -- start 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 -- start start 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14103450 0x7f9b14075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14108a80 0x7f9b140757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112c10 con 0x7f9b14102a90 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112a90 con 0x7f9b14103450 2024-04-25T21:03:59.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b18ccb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9b14112d90 con 0x7f9b14108a80 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50994/0 (socket says 172.21.15.37:50994) 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 -- 172.21.15.37:0/1386531941 learned_addr learned my addr 172.21.15.37:0/1386531941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b11d9b700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14103450 0x7f9b14075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b12d9d700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14108a80 0x7f9b140757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 -- 172.21.15.37:0/1386531941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14108a80 msgr2=0x7f9b140757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14108a80 0x7f9b140757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 -- 172.21.15.37:0/1386531941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14103450 msgr2=0x7f9b14075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14103450 0x7f9b14075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.187+0000 7f9b1259c700 1 -- 172.21.15.37:0/1386531941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141a75a0 con 0x7f9b14102a90 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b1259c700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04027070 con 0x7f9b14102a90 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b040040e0 con 0x7f9b14102a90 2024-04-25T21:03:59.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b141a7830 con 0x7f9b14102a90 2024-04-25T21:03:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400f460 con 0x7f9b14102a90 2024-04-25T21:03:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.188+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b141a7d10 con 0x7f9b14102a90 2024-04-25T21:03:59.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.189+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b14069730 con 0x7f9b14102a90 2024-04-25T21:03:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.190+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b0402a070 con 0x7f9b14102a90 2024-04-25T21:03:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.191+0000 7f9b037fe700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 0x7f9afc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.191+0000 7f9b11d9b700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 0x7f9afc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.191+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9b040d1370 con 0x7f9b14102a90 2024-04-25T21:03:59.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.191+0000 7f9b11d9b700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 0x7f9afc0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b1407a210 tx=0x7f9b0800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:59.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.194+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b04031070 con 0x7f9b14102a90 2024-04-25T21:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.402+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b1402df80 con 0x7f9b14102a90 2024-04-25T21:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.403+0000 7f9b037fe700 1 -- 172.21.15.37:0/1386531941 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18373 (secure 0 0 0) 0x7f9b04013070 con 0x7f9b14102a90 2024-04-25T21:03:59.403 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:03:59.404 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:03:54.603747+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:03:59.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 msgr2=0x7f9afc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 0x7f9afc0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b1407a210 tx=0x7f9b0800a380 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 msgr2=0x7f9b14074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9b04000c00 tx=0x7f9b040047b0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 shutdown_connections 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9b14108a80 0x7f9b140757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b14102a90 0x7f9b14074d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9afc0a3c40 0x7f9afc0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 --2- 172.21.15.37:0/1386531941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b14103450 0x7f9b14075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 >> 172.21.15.37:0/1386531941 conn(0x7f9b14079bf0 msgr2=0x7f9b14101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 shutdown_connections 2024-04-25T21:03:59.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:03:59.405+0000 7f9b18ccb700 1 -- 172.21.15.37:0/1386531941 wait complete. 2024-04-25T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:59 smithi037 ceph-mon[29195]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:59.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:03:59 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1386531941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:03:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:59 smithi098 ceph-mon[32569]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:03:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:03:59 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1386531941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:04:00.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:59 smithi177 ceph-mon[33043]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:00.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:03:59 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1386531941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:04:00.254 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T21:00:00.177409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T21:04:00.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T21:04:00.505 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:01 smithi098 ceph-mon[32569]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:01.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:01 smithi037 ceph-mon[29195]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:01 smithi177 ceph-mon[33043]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 -- 172.21.15.37:0/1615238396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a810f4f0 msgr2=0x7f60a81118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 --2- 172.21.15.37:0/1615238396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a810f4f0 0x7f60a81118e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6098009990 tx=0x7f6098009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 -- 172.21.15.37:0/1615238396 shutdown_connections 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 --2- 172.21.15.37:0/1615238396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a810f4f0 0x7f60a81118e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 --2- 172.21.15.37:0/1615238396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806a750 0x7f60a806ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 --2- 172.21.15.37:0/1615238396 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a806b070 0x7f60a810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 -- 172.21.15.37:0/1615238396 >> 172.21.15.37:0/1615238396 conn(0x7f60a80788f0 msgr2=0x7f60a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 -- 172.21.15.37:0/1615238396 shutdown_connections 2024-04-25T21:04:02.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.306+0000 7f60af277700 1 -- 172.21.15.37:0/1615238396 wait complete. 2024-04-25T21:04:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 Processor -- start 2024-04-25T21:04:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 -- start start 2024-04-25T21:04:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:02.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806b070 0x7f60a819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 0x7f60a819fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8114df0 con 0x7f60a806a750 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8114c70 con 0x7f60a806b070 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60af277700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8114f70 con 0x7f60a810f4f0 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51014/0 (socket says 172.21.15.37:51014) 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 -- 172.21.15.37:0/3007941040 learned_addr learned my addr 172.21.15.37:0/3007941040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ac812700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806b070 0x7f60a819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad814700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 0x7f60a819fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 -- 172.21.15.37:0/3007941040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 msgr2=0x7f60a819fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 0x7f60a819fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 -- 172.21.15.37:0/3007941040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806b070 msgr2=0x7f60a819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:02.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806b070 0x7f60a819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.307+0000 7f60ad013700 1 -- 172.21.15.37:0/3007941040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81a0240 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60ad814700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 0x7f60a819fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60ad013700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007e20 tx=0x7f60a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0016020 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a000ae00 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0011700 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a0530 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.308+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a7db0 con 0x7f60a806a750 2024-04-25T21:04:02.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.309+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8069730 con 0x7f60a806a750 2024-04-25T21:04:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.310+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60a0019020 con 0x7f60a806a750 2024-04-25T21:04:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.311+0000 7f60a67fc700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 0x7f60900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.311+0000 7f60ac812700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 0x7f60900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.311+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f60a00d00d0 con 0x7f60a806a750 2024-04-25T21:04:02.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.311+0000 7f60ac812700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 0x7f60900a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60a8102d50 tx=0x7f609c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:02.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.315+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60a00182c0 con 0x7f60a806a750 2024-04-25T21:04:02.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.522+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f60a802df80 con 0x7f60a806a750 2024-04-25T21:04:02.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.522+0000 7f60a67fc700 1 -- 172.21.15.37:0/3007941040 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f60a009cf50 con 0x7f60a806a750 2024-04-25T21:04:02.522 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 msgr2=0x7f60900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 0x7f60900a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f60a8102d50 tx=0x7f609c00b590 comp rx=0 tx=0).stop 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 msgr2=0x7f60a819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60a0007e20 tx=0x7f60a000a340 comp rx=0 tx=0).stop 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 shutdown_connections 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a810f4f0 0x7f60a819fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60a806a750 0x7f60a819b280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f60900a3c40 0x7f60900a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 --2- 172.21.15.37:0/3007941040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f60a806b070 0x7f60a819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.526+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 >> 172.21.15.37:0/3007941040 conn(0x7f60a80788f0 msgr2=0x7f60a8110940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.527+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 shutdown_connections 2024-04-25T21:04:02.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:02.527+0000 7f60af277700 1 -- 172.21.15.37:0/3007941040 wait complete. 2024-04-25T21:04:02.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:02 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3007941040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:04:02.951 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:02 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3007941040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:04:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:02 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3007941040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:04:03.336 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.b on smithi098 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.e on smithi098 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.h on smithi098 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.k on smithi098 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.c on smithi177 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.f on smithi177 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.i on smithi177 2024-04-25T21:04:03.337 INFO:tasks.cephadm:Adding mds.l on smithi177 2024-04-25T21:04:03.338 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l' 2024-04-25T21:04:03.586 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:04:03.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:03 smithi098 ceph-mon[32569]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:03.959 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:03 smithi177 ceph-mon[33043]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:03.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:03 smithi037 ceph-mon[29195]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:05.248 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.246+0000 7fe252dc9700 1 -- 172.21.15.177:0/1939900208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c102aa0 msgr2=0x7fe24c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.246+0000 7fe252dc9700 1 --2- 172.21.15.177:0/1939900208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c102aa0 0x7fe24c102ec0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe240009990 tx=0x7fe240009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- 172.21.15.177:0/1939900208 shutdown_connections 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- 172.21.15.177:0/1939900208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c104640 0x7fe24c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- 172.21.15.177:0/1939900208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c102aa0 0x7fe24c102ec0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- 172.21.15.177:0/1939900208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c103ca0 0x7fe24c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- 172.21.15.177:0/1939900208 >> 172.21.15.177:0/1939900208 conn(0x7fe24c0fe040 msgr2=0x7fe24c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- 172.21.15.177:0/1939900208 shutdown_connections 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- 172.21.15.177:0/1939900208 wait complete. 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 Processor -- start 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- start start 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 0x7fe24c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:05.249 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c103ca0 0x7fe24c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10e610 con 0x7fe24c103ca0 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10e490 con 0x7fe24c102aa0 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe252dc9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe24c10e790 con 0x7fe24c104640 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe251366700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe251366700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:55998/0 (socket says 172.21.15.177:55998) 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.247+0000 7fe251366700 1 -- 172.21.15.177:0/2391389406 learned_addr learned my addr 172.21.15.177:0/2391389406 (peer_addr_for_me v2:172.21.15.177:0/0) 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe250b65700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 0x7fe24c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe24bfff700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c103ca0 0x7fe24c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 -- 172.21.15.177:0/2391389406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 msgr2=0x7fe24c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 0x7fe24c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.250 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 -- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c103ca0 msgr2=0x7fe24c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c103ca0 0x7fe24c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 -- 172.21.15.177:0/2391389406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24c1a0950 con 0x7fe24c104640 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe250b65700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 0x7fe24c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe251366700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe24400b970 tx=0x7fe24400bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe244019070 con 0x7fe24c104640 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe244014440 con 0x7fe24c104640 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe24c1a0c40 con 0x7fe24c104640 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24400ce60 con 0x7fe24c104640 2024-04-25T21:04:05.251 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.248+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe24c1a1130 con 0x7fe24c104640 2024-04-25T21:04:05.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.249+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe24c069730 con 0x7fe24c104640 2024-04-25T21:04:05.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.250+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe24401c070 con 0x7fe24c104640 2024-04-25T21:04:05.252 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.251+0000 7fe249ffb700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 0x7fe23c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:05.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.251+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe2440cf3b0 con 0x7fe24c104640 2024-04-25T21:04:05.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.251+0000 7fe250b65700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 0x7fe23c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:05.253 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.252+0000 7fe250b65700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 0x7fe23c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe240003790 tx=0x7fe240005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:05.256 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.256+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe24409c230 con 0x7fe24c104640 2024-04-25T21:04:05.473 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.472+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}) -- 0x7fe24c063080 con 0x7fe23c0a3c40 2024-04-25T21:04:05.481 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.481+0000 7fe249ffb700 1 -- 172.21.15.177:0/2391389406 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe24c063080 con 0x7fe23c0a3c40 2024-04-25T21:04:05.481 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mds.all update... 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 msgr2=0x7fe23c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 0x7fe23c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe240003790 tx=0x7fe240005fd0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 msgr2=0x7fe24c1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe24400b970 tx=0x7fe24400bd30 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 shutdown_connections 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe24c104640 0x7fe24c1a0280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe23c0a3c40 0x7fe23c0a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe24c102aa0 0x7fe24c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 --2- 172.21.15.177:0/2391389406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe24c103ca0 0x7fe24c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.483+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 >> 172.21.15.177:0/2391389406 conn(0x7fe24c0fe040 msgr2=0x7fe24c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.484+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 shutdown_connections 2024-04-25T21:04:05.484 INFO:teuthology.orchestra.run.smithi177.stderr:2024-04-25T21:04:05.484+0000 7fe252dc9700 1 -- 172.21.15.177:0/2391389406 wait complete. 2024-04-25T21:04:05.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:05 smithi177 ceph-mon[33043]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:05.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:05.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:05 smithi098 ceph-mon[32569]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:05 smithi037 ceph-mon[29195]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:06.195 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T21:04:06.198 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T21:04:06.241 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T21:04:06.284 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T21:04:06.327 DEBUG:teuthology.orchestra.run.smithi098:mds.b> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T21:04:06.330 DEBUG:teuthology.orchestra.run.smithi098:mds.e> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T21:04:06.372 DEBUG:teuthology.orchestra.run.smithi098:mds.h> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T21:04:06.415 DEBUG:teuthology.orchestra.run.smithi098:mds.k> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T21:04:06.458 DEBUG:teuthology.orchestra.run.smithi177:mds.c> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T21:04:06.461 DEBUG:teuthology.orchestra.run.smithi177:mds.f> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T21:04:06.504 DEBUG:teuthology.orchestra.run.smithi177:mds.i> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T21:04:06.547 DEBUG:teuthology.orchestra.run.smithi177:mds.l> sudo journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T21:04:06.590 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T21:04:06.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:06.761 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:06 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:06.846 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:06.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:06.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:06.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:06 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:06.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:06.969 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:06 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:07.799 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:07 smithi177 ceph-mon[33043]: Deploying daemon mds.l on smithi177 2024-04-25T21:04:07.799 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:07 smithi177 ceph-mon[33043]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:07 smithi098 ceph-mon[32569]: Deploying daemon mds.l on smithi177 2024-04-25T21:04:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:07 smithi098 ceph-mon[32569]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:07 smithi037 ceph-mon[29195]: Deploying daemon mds.l on smithi177 2024-04-25T21:04:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:07 smithi037 ceph-mon[29195]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:08.340 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 systemd[1]: Starting Ceph mds.l for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:08.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 -- 172.21.15.37:0/3327897764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 msgr2=0x7f49d4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 --2- 172.21.15.37:0/3327897764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d4102f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009990 tx=0x7f49c8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 -- 172.21.15.37:0/3327897764 shutdown_connections 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 --2- 172.21.15.37:0/3327897764 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 0x7f49d410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 --2- 172.21.15.37:0/3327897764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 0x7f49d4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 --2- 172.21.15.37:0/3327897764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d4102f40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 -- 172.21.15.37:0/3327897764 >> 172.21.15.37:0/3327897764 conn(0x7f49d4079bb0 msgr2=0x7f49d4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 -- 172.21.15.37:0/3327897764 shutdown_connections 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 -- 172.21.15.37:0/3327897764 wait complete. 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.638+0000 7f49da906700 1 Processor -- start 2024-04-25T21:04:08.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 -- start start 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 0x7f49d4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 0x7f49d419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4112ca0 con 0x7f49d4102ac0 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4112b20 con 0x7f49d4108a70 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49da906700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49d4112e20 con 0x7f49d4103480 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36610/0 (socket says 172.21.15.37:36610) 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d3fff700 1 -- 172.21.15.37:0/4101937279 learned_addr learned my addr 172.21.15.37:0/4101937279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d37fe700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 0x7f49d4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:08.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d8ea3700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 0x7f49d419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.639+0000 7f49d3fff700 1 -- 172.21.15.37:0/4101937279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 msgr2=0x7f49d4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.640+0000 7f49d3fff700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 0x7f49d4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.640+0000 7f49d3fff700 1 -- 172.21.15.37:0/4101937279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 msgr2=0x7f49d419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.640+0000 7f49d3fff700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 0x7f49d419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.640+0000 7f49d3fff700 1 -- 172.21.15.37:0/4101937279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d419e1b0 con 0x7f49d4102ac0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.640+0000 7f49d3fff700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f49c400e9a0 tx=0x7f49c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.641+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c400cbd0 con 0x7f49d4102ac0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.641+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49c400cd70 con 0x7f49d4102ac0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.641+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d419e4a0 con 0x7f49d4102ac0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.641+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4018680 con 0x7f49d4102ac0 2024-04-25T21:04:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.641+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d41a5d20 con 0x7f49d4102ac0 2024-04-25T21:04:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.642+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d4069730 con 0x7f49d4102ac0 2024-04-25T21:04:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.643+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f49c4010430 con 0x7f49d4102ac0 2024-04-25T21:04:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.643+0000 7f49d17fa700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 0x7f49bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.644+0000 7f49d37fe700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 0x7f49bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.644+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f49c40d0250 con 0x7f49d4102ac0 2024-04-25T21:04:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.644+0000 7f49d37fe700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 0x7f49bc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49d407a220 tx=0x7f49c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:08.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.648+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49c409d030 con 0x7f49d4102ac0 2024-04-25T21:04:08.779 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 podman[62802]: 2024-04-25 21:04:08.387277199 +0000 UTC m=+0.018303422 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:08.779 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 podman[62802]: 2024-04-25 21:04:08.501827467 +0000 UTC m=+0.132853679 container create 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:04:08.779 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 podman[62802]: 2024-04-25 21:04:08.779697138 +0000 UTC m=+0.410723351 container init 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:04:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.903+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49d402df80 con 0x7f49d4102ac0 2024-04-25T21:04:08.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.903+0000 7f49d17fa700 1 -- 172.21.15.37:0/4101937279 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f49c409c930 con 0x7f49d4102ac0 2024-04-25T21:04:08.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:08.903 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T21:04:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.905+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 msgr2=0x7f49bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.905+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 0x7f49bc0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49d407a220 tx=0x7f49c801a040 comp rx=0 tx=0).stop 2024-04-25T21:04:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.905+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 msgr2=0x7f49d41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.905+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f49c400e9a0 tx=0x7f49c400ecb0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 shutdown_connections 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49bc0a3c40 0x7f49bc0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49d4108a70 0x7f49d419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49d4102ac0 0x7f49d41991f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 --2- 172.21.15.37:0/4101937279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49d4103480 0x7f49d4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 >> 172.21.15.37:0/4101937279 conn(0x7f49d4079bb0 msgr2=0x7f49d4101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 shutdown_connections 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:08.906+0000 7f49da906700 1 -- 172.21.15.37:0/4101937279 wait complete. 2024-04-25T21:04:08.906 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-25T21:04:09.089 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 podman[62802]: 2024-04-25 21:04:08.782443075 +0000 UTC m=+0.413469289 container start 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:04:09.090 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 ceph-mds[62821]: -- 172.21.15.177:0/1880170657 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a99449a40 con 0x561a9941d000 2024-04-25T21:04:09.090 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: starting mds.l at 2024-04-25T21:04:09.090 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 bash[62802]: 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f 2024-04-25T21:04:09.090 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:04:08 smithi177 systemd[1]: Started Ceph mds.l for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:09.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:09.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6832/3715259407,v1:172.21.15.177:6833/3715259407] up:boot 2024-04-25T21:04:09.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: fsmap 1 up:standby 2024-04-25T21:04:09.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:04:09.910 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4101937279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:09.911 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:09 smithi098 ceph-mon[32569]: Deploying daemon mds.k on smithi098 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6832/3715259407,v1:172.21.15.177:6833/3715259407] up:boot 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: fsmap 1 up:standby 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4101937279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:09.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:09.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:09.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:09.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:09 smithi037 ceph-mon[29195]: Deploying daemon mds.k on smithi098 2024-04-25T21:04:10.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:10.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6832/3715259407,v1:172.21.15.177:6833/3715259407] up:boot 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: fsmap 1 up:standby 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4101937279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:10.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:09 smithi177 ceph-mon[33043]: Deploying daemon mds.k on smithi098 2024-04-25T21:04:11.417 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 systemd[1]: Starting Ceph mds.k for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:11.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:11.762 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 podman[62008]: 2024-04-25 21:04:11.376587302 +0000 UTC m=+0.017607364 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:11.763 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 podman[62008]: 2024-04-25 21:04:11.502011055 +0000 UTC m=+0.143031111 container create e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:04:11.763 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 podman[62008]: 2024-04-25 21:04:11.763025465 +0000 UTC m=+0.404045513 container init e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:04:11.951 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:11 smithi037 ceph-mon[29195]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:12.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:11 smithi177 ceph-mon[33043]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:11 smithi098 ceph-mon[32569]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:12.168 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 podman[62008]: 2024-04-25 21:04:11.765851837 +0000 UTC m=+0.406871896 container start e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True) 2024-04-25T21:04:12.168 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 ceph-mds[62027]: -- 172.21.15.98:0/2249096478 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d163625a40 con 0x55d1635f8c00 2024-04-25T21:04:12.168 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: starting mds.k at 2024-04-25T21:04:12.168 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 bash[62008]: e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 2024-04-25T21:04:12.168 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:04:11 smithi098 systemd[1]: Started Ceph mds.k for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6832/3425764534,v1:172.21.15.98:6833/3425764534] up:boot 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: fsmap 2 up:standby 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:12 smithi177 ceph-mon[33043]: Deploying daemon mds.j on smithi037 2024-04-25T21:04:13.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6832/3425764534,v1:172.21.15.98:6833/3425764534] up:boot 2024-04-25T21:04:13.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: fsmap 2 up:standby 2024-04-25T21:04:13.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:04:13.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.099 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:13.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:13.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:13.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:12 smithi037 ceph-mon[29195]: Deploying daemon mds.j on smithi037 2024-04-25T21:04:13.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6832/3425764534,v1:172.21.15.98:6833/3425764534] up:boot 2024-04-25T21:04:13.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: fsmap 2 up:standby 2024-04-25T21:04:13.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:13.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:12 smithi098 ceph-mon[32569]: Deploying daemon mds.j on smithi037 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.938+0000 7f57d9dc5700 1 -- 172.21.15.37:0/3191934843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d406b730 msgr2=0x7f57d406bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.938+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/3191934843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d406b730 0x7f57d406bbb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009990 tx=0x7f57c8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 -- 172.21.15.37:0/3191934843 shutdown_connections 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/3191934843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 0x7f57d4105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/3191934843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d41079a0 0x7f57d406b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/3191934843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d406b730 0x7f57d406bbb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 -- 172.21.15.37:0/3191934843 >> 172.21.15.37:0/3191934843 conn(0x7f57d40789e0 msgr2=0x7f57d4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 -- 172.21.15.37:0/3191934843 shutdown_connections 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 -- 172.21.15.37:0/3191934843 wait complete. 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 Processor -- start 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.939+0000 7f57d9dc5700 1 -- start start 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 0x7f57d41a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 0x7f57d41a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f57d41192a0 con 0x7f57d41079a0 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4119120 con 0x7f57d41054b0 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d9dc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f57d4119420 con 0x7f57d406b730 2024-04-25T21:04:13.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36630/0 (socket says 172.21.15.37:36630) 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 -- 172.21.15.37:0/341255037 learned_addr learned my addr 172.21.15.37:0/341255037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d2ffd700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 0x7f57d41a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d37fe700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 0x7f57d41a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 -- 172.21.15.37:0/341255037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 msgr2=0x7f57d41a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 0x7f57d41a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 -- 172.21.15.37:0/341255037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 msgr2=0x7f57d41a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 0x7f57d41a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 -- 172.21.15.37:0/341255037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d419fd60 con 0x7f57d41079a0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d37fe700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 0x7f57d41a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.940+0000 7f57d3fff700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007ee0 tx=0x7f57cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.941+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc00ddd0 con 0x7f57d41079a0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.941+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc004020 con 0x7f57d41079a0 2024-04-25T21:04:13.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.941+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d41a0050 con 0x7f57d41079a0 2024-04-25T21:04:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.941+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc015660 con 0x7f57d41079a0 2024-04-25T21:04:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.941+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d41ac2c0 con 0x7f57d41079a0 2024-04-25T21:04:13.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.942+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d4069730 con 0x7f57d41079a0 2024-04-25T21:04:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.943+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f57cc0041d0 con 0x7f57d41079a0 2024-04-25T21:04:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.943+0000 7f57d0ff9700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 0x7f57bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.944+0000 7f57d37fe700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 0x7f57bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.944+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f57cc0d01e0 con 0x7f57d41079a0 2024-04-25T21:04:13.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.944+0000 7f57d37fe700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 0x7f57bc0a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57c40099d0 tx=0x7f57c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:13.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:13.947+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57cc010070 con 0x7f57d41079a0 2024-04-25T21:04:14.183 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:13 smithi098 ceph-mon[32569]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.206+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57d41ac490 con 0x7f57d41079a0 2024-04-25T21:04:14.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:13 smithi037 ceph-mon[29195]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:14.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.207+0000 7f57d0ff9700 1 -- 172.21.15.37:0/341255037 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f57cc09d060 con 0x7f57d41079a0 2024-04-25T21:04:14.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:14.208 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T21:04:14.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 msgr2=0x7f57bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:14.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 0x7f57bc0a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f57c40099d0 tx=0x7f57c4008040 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 msgr2=0x7f57d419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007ee0 tx=0x7f57cc00d3b0 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 shutdown_connections 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57d41079a0 0x7f57d419f600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57d406b730 0x7f57d41a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f57bc0a3c40 0x7f57bc0a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 --2- 172.21.15.37:0/341255037 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f57d41054b0 0x7f57d41a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 >> 172.21.15.37:0/341255037 conn(0x7f57d40789e0 msgr2=0x7f57d4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 shutdown_connections 2024-04-25T21:04:14.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:14.209+0000 7f57d9dc5700 1 -- 172.21.15.37:0/341255037 wait complete. 2024-04-25T21:04:14.211 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-25T21:04:14.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:13 smithi177 ceph-mon[33043]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:15.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:14 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/341255037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:15.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:14 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/341255037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:15.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:14 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/341255037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:16.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:15 smithi037 ceph-mon[29195]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:16.224 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 systemd[1]: Starting Ceph mds.j for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:16.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:15 smithi177 ceph-mon[33043]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:16.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:15 smithi098 ceph-mon[32569]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:16.484 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 podman[70409]: 2024-04-25 21:04:16.127117852 +0000 UTC m=+0.018558448 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:16.485 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 podman[70409]: 2024-04-25 21:04:16.248541994 +0000 UTC m=+0.139982579 container create 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T21:04:16.485 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 podman[70409]: 2024-04-25 21:04:16.459387337 +0000 UTC m=+0.350827926 container init 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:04:16.485 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 podman[70409]: 2024-04-25 21:04:16.462066833 +0000 UTC m=+0.353507428 container start 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T21:04:16.735 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 ceph-mds[70428]: -- 172.21.15.37:0/3050287924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605b0d77a40 con 0x5605b0d4b000 2024-04-25T21:04:16.735 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j[70424]: starting mds.j at 2024-04-25T21:04:16.735 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 bash[70409]: 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec 2024-04-25T21:04:16.735 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:04:16 smithi037 systemd[1]: Started Ceph mds.j for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:16.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: Deploying daemon mds.d on smithi037 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6834/719027051,v1:172.21.15.37:6835/719027051] up:boot 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: fsmap 3 up:standby 2024-04-25T21:04:17.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:17 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: Deploying daemon mds.d on smithi037 2024-04-25T21:04:17.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6834/719027051,v1:172.21.15.37:6835/719027051] up:boot 2024-04-25T21:04:17.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: fsmap 3 up:standby 2024-04-25T21:04:17.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:17 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: Deploying daemon mds.d on smithi037 2024-04-25T21:04:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6834/719027051,v1:172.21.15.37:6835/719027051] up:boot 2024-04-25T21:04:18.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: fsmap 3 up:standby 2024-04-25T21:04:18.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:17 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:04:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:19.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:19 smithi037 ceph-mon[29195]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:19.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:19 smithi098 ceph-mon[32569]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:19 smithi177 ceph-mon[33043]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 -- 172.21.15.37:0/1083834066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 msgr2=0x7f77e4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 --2- 172.21.15.37:0/1083834066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4107d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009990 tx=0x7f77e0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 -- 172.21.15.37:0/1083834066 shutdown_connections 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 --2- 172.21.15.37:0/1083834066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 0x7f77e410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 --2- 172.21.15.37:0/1083834066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4107d70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 --2- 172.21.15.37:0/1083834066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 0x7f77e4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 -- 172.21.15.37:0/1083834066 >> 172.21.15.37:0/1083834066 conn(0x7f77e40788f0 msgr2=0x7f77e4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 -- 172.21.15.37:0/1083834066 shutdown_connections 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.217+0000 7f77ec528700 1 -- 172.21.15.37:0/1083834066 wait complete. 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 Processor -- start 2024-04-25T21:04:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 -- start start 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 0x7f77e41034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 0x7f77e4103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4111bd0 con 0x7f77e4107990 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4111a50 con 0x7f77e4102300 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ec528700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f77e4111d50 con 0x7f77e4101940 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77eaac5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77eaac5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60338/0 (socket says 172.21.15.37:60338) 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77ea2c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 0x7f77e41034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.218+0000 7f77eaac5700 1 -- 172.21.15.37:0/194313369 learned_addr learned my addr 172.21.15.37:0/194313369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77e9ac3700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 0x7f77e4103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 -- 172.21.15.37:0/194313369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 msgr2=0x7f77e41034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 0x7f77e41034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 -- 172.21.15.37:0/194313369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 msgr2=0x7f77e4103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 0x7f77e4103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 -- 172.21.15.37:0/194313369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e41a77a0 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77eaac5700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b970 tx=0x7f77dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00d860 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77dc00e430 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e41a7a90 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc017660 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.219+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e41a7f80 con 0x7f77e4107990 2024-04-25T21:04:20.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.220+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e4069730 con 0x7f77e4107990 2024-04-25T21:04:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.222+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f77dc00da00 con 0x7f77e4107990 2024-04-25T21:04:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.222+0000 7f77db7fe700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.223+0000 7f77ea2c4700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.223+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f77dc0cfd00 con 0x7f77e4107990 2024-04-25T21:04:20.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.223+0000 7f77ea2c4700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 0x7f77d00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b5c0 tx=0x7f77e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.226+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77dc013030 con 0x7f77e4107990 2024-04-25T21:04:20.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.491+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77e41a82e0 con 0x7f77e4107990 2024-04-25T21:04:20.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.491+0000 7f77db7fe700 1 -- 172.21.15.37:0/194313369 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f77dc09cb80 con 0x7f77e4107990 2024-04-25T21:04:20.492 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:20 smithi037 podman[70837]: 2024-04-25 21:04:20.398704144 +0000 UTC m=+0.384134714 container create bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T21:04:20.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:20.493 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 msgr2=0x7f77d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 0x7f77d00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b5c0 tx=0x7f77e0005960 comp rx=0 tx=0).stop 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 msgr2=0x7f77e4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b970 tx=0x7f77dc00bc80 comp rx=0 tx=0).stop 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 shutdown_connections 2024-04-25T21:04:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77e4107990 0x7f77e4103f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f77e4101940 0x7f77e41034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f77d00a3c40 0x7f77d00a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.493+0000 7f77ec528700 1 --2- 172.21.15.37:0/194313369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77e4102300 0x7f77e4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.494+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 >> 172.21.15.37:0/194313369 conn(0x7f77e40788f0 msgr2=0x7f77e406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.494+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 shutdown_connections 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:20.494+0000 7f77ec528700 1 -- 172.21.15.37:0/194313369 wait complete. 2024-04-25T21:04:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-04-25T21:04:20.787 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:20 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/194313369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:20.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:20 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/194313369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:20 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/194313369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 podman[70837]: 2024-04-25 21:04:21.282236004 +0000 UTC m=+1.267666590 container init bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 podman[70837]: 2024-04-25 21:04:21.285602013 +0000 UTC m=+1.271032583 container start bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mds[70907]: -- 172.21.15.37:0/1383212658 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624c2507a40 con 0x5624c24da800 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d[70889]: starting mds.d at 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 bash[70837]: bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 2024-04-25T21:04:21.485 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:04:21 smithi037 systemd[1]: Started Ceph mds.d for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:21.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:21.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:21 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:21.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:21 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:22.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:21 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:22.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:22 smithi098 ceph-mon[32569]: Deploying daemon mds.e on smithi098 2024-04-25T21:04:22.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:22 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6836/1767596053,v1:172.21.15.37:6837/1767596053] up:boot 2024-04-25T21:04:22.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:22 smithi098 ceph-mon[32569]: fsmap 4 up:standby 2024-04-25T21:04:22.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:22 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:04:22.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:22 smithi037 ceph-mon[29195]: Deploying daemon mds.e on smithi098 2024-04-25T21:04:22.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:22 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6836/1767596053,v1:172.21.15.37:6837/1767596053] up:boot 2024-04-25T21:04:22.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:22 smithi037 ceph-mon[29195]: fsmap 4 up:standby 2024-04-25T21:04:22.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:22 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:04:23.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:22 smithi177 ceph-mon[33043]: Deploying daemon mds.e on smithi098 2024-04-25T21:04:23.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:22 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6836/1767596053,v1:172.21.15.37:6837/1767596053] up:boot 2024-04-25T21:04:23.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:22 smithi177 ceph-mon[33043]: fsmap 4 up:standby 2024-04-25T21:04:23.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:22 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:04:23.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:23.473 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:23.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:23 smithi037 ceph-mon[29195]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:23.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:23 smithi098 ceph-mon[32569]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:23.918 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:23 smithi098 systemd[1]: Starting Ceph mds.e for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:24.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:23 smithi177 ceph-mon[33043]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:24.294 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 podman[62325]: 2024-04-25 21:04:23.920264683 +0000 UTC m=+0.018043887 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:24.294 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 podman[62325]: 2024-04-25 21:04:24.050925636 +0000 UTC m=+0.148704855 container create 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:04:24.294 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 podman[62325]: 2024-04-25 21:04:24.295184677 +0000 UTC m=+0.392963854 container init 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:04:24.603 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 podman[62325]: 2024-04-25 21:04:24.297932212 +0000 UTC m=+0.395711394 container start 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T21:04:24.604 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 ceph-mds[62344]: -- 172.21.15.98:0/4294769701 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556cd1e61a40 con 0x556cd1e34c00 2024-04-25T21:04:24.604 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: starting mds.e at 2024-04-25T21:04:24.604 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 bash[62325]: 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf 2024-04-25T21:04:24.604 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:04:24 smithi098 systemd[1]: Started Ceph mds.e for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:25.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 -- 172.21.15.37:0/228001266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518101940 msgr2=0x7f3518101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 --2- 172.21.15.37:0/228001266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518101940 0x7f3518101dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 -- 172.21.15.37:0/228001266 shutdown_connections 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 --2- 172.21.15.37:0/228001266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518102300 0x7f351810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 --2- 172.21.15.37:0/228001266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 0x7f3518107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 --2- 172.21.15.37:0/228001266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518101940 0x7f3518101dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 -- 172.21.15.37:0/228001266 >> 172.21.15.37:0/228001266 conn(0x7f35180788f0 msgr2=0x7f3518078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 -- 172.21.15.37:0/228001266 shutdown_connections 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.121+0000 7f351e3a1700 1 -- 172.21.15.37:0/228001266 wait complete. 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 Processor -- start 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 -- start start 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:25.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518102300 0x7f35181996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 0x7f351819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111b00 con 0x7f3518102300 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111980 con 0x7f3518101940 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.122+0000 7f351e3a1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111c80 con 0x7f3518107990 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:38696/0 (socket says 172.21.15.37:38696) 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 -- 172.21.15.37:0/894842966 learned_addr learned my addr 172.21.15.37:0/894842966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f351c93e700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 0x7f351819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f35177fe700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518102300 0x7f35181996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 -- 172.21.15.37:0/894842966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 msgr2=0x7f351819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 0x7f351819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 -- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518102300 msgr2=0x7f35181996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518102300 0x7f35181996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 -- 172.21.15.37:0/894842966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351819e140 con 0x7f3518101940 2024-04-25T21:04:25.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f3517fff700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f350800d980 tx=0x7f350800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508017070 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351819e430 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.123+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3508005020 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.124+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350800fa10 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.124+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35181a5cb0 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.124+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3518069730 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.126+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3508004a30 con 0x7f3518101940 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.126+0000 7f35157fa700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 0x7f35040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:25.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.127+0000 7f35177fe700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 0x7f35040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:25.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.127+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35080d01f0 con 0x7f3518101940 2024-04-25T21:04:25.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.127+0000 7f35177fe700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 0x7f35040a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35180790f0 tx=0x7f350000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:25.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.130+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3508020020 con 0x7f3518101940 2024-04-25T21:04:25.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.384+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f351802df80 con 0x7f3518101940 2024-04-25T21:04:25.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.385+0000 7f35157fa700 1 -- 172.21.15.37:0/894842966 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f350809cfd0 con 0x7f3518101940 2024-04-25T21:04:25.385 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:25.385 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T21:04:25.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 msgr2=0x7f35040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:25.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 0x7f35040a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35180790f0 tx=0x7f350000b280 comp rx=0 tx=0).stop 2024-04-25T21:04:25.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 msgr2=0x7f3518199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:25.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f350800d980 tx=0x7f350800dc90 comp rx=0 tx=0).stop 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 shutdown_connections 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3518107990 0x7f351819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f35040a3c40 0x7f35040a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3518101940 0x7f3518199180 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 --2- 172.21.15.37:0/894842966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3518102300 0x7f35181996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 >> 172.21.15.37:0/894842966 conn(0x7f35180788f0 msgr2=0x7f351806aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 shutdown_connections 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:25.387+0000 7f351e3a1700 1 -- 172.21.15.37:0/894842966 wait complete. 2024-04-25T21:04:25.388 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: Deploying daemon mds.f on smithi177 2024-04-25T21:04:25.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6834/785792135,v1:172.21.15.98:6835/785792135] up:boot 2024-04-25T21:04:25.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: fsmap 5 up:standby 2024-04-25T21:04:25.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:04:25.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:25 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/894842966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:25.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:25.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: Deploying daemon mds.f on smithi177 2024-04-25T21:04:25.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6834/785792135,v1:172.21.15.98:6835/785792135] up:boot 2024-04-25T21:04:25.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: fsmap 5 up:standby 2024-04-25T21:04:25.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:04:25.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:25 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/894842966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:25.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: Deploying daemon mds.f on smithi177 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6834/785792135,v1:172.21.15.98:6835/785792135] up:boot 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: fsmap 5 up:standby 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:04:25.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:25 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/894842966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:26.841 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:26 smithi177 systemd[1]: Starting Ceph mds.f for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:27.226 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:26 smithi177 podman[63111]: 2024-04-25 21:04:26.856266255 +0000 UTC m=+0.018134620 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:27.226 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:26 smithi177 podman[63111]: 2024-04-25 21:04:26.981312363 +0000 UTC m=+0.143180700 container create c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T21:04:27.226 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 podman[63111]: 2024-04-25 21:04:27.20071885 +0000 UTC m=+0.362587188 container init c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:04:27.226 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 podman[63111]: 2024-04-25 21:04:27.203253573 +0000 UTC m=+0.365121912 container start c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mds[63130]: -- 172.21.15.177:0/3122596130 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560802aa5a40 con 0x560802a78c00 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: starting mds.f at 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 bash[63111]: c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:04:27 smithi177 systemd[1]: Started Ceph mds.f for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:27 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:27.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:27.736 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:27 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:27 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:28.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:28.319 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:28.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:28 smithi037 ceph-mon[29195]: Deploying daemon mds.c on smithi177 2024-04-25T21:04:28.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:28 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:boot 2024-04-25T21:04:28.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:28 smithi037 ceph-mon[29195]: fsmap 6 up:standby 2024-04-25T21:04:28.717 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:28 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:04:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:28 smithi177 ceph-mon[33043]: Deploying daemon mds.c on smithi177 2024-04-25T21:04:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:28 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:boot 2024-04-25T21:04:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:28 smithi177 ceph-mon[33043]: fsmap 6 up:standby 2024-04-25T21:04:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:28 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:04:28.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:28 smithi098 ceph-mon[32569]: Deploying daemon mds.c on smithi177 2024-04-25T21:04:28.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:28 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:boot 2024-04-25T21:04:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:28 smithi098 ceph-mon[32569]: fsmap 6 up:standby 2024-04-25T21:04:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:28 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:04:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:29 smithi177 ceph-mon[33043]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:29.590 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 systemd[1]: Starting Ceph mds.c for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:29.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:29 smithi037 ceph-mon[29195]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:29 smithi098 ceph-mon[32569]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:29.964 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 podman[63420]: 2024-04-25 21:04:29.590364832 +0000 UTC m=+0.018620657 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:29.964 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 podman[63420]: 2024-04-25 21:04:29.712435566 +0000 UTC m=+0.140691423 container create 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:04:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3330566231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 msgr2=0x7fe598101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3330566231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe598101db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009990 tx=0x7fe58c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3330566231 shutdown_connections 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3330566231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5981022f0 0x7fe59810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3330566231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe598107980 0x7fe598107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3330566231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe598101db0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3330566231 >> 172.21.15.37:0/3330566231 conn(0x7fe5980788b0 msgr2=0x7fe598078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3330566231 shutdown_connections 2024-04-25T21:04:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.982+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3330566231 wait complete. 2024-04-25T21:04:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.983+0000 7fe59cdc5700 1 Processor -- start 2024-04-25T21:04:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.983+0000 7fe59cdc5700 1 -- start start 2024-04-25T21:04:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5981022f0 0x7fe59819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe598107980 0x7fe59819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111ad0 con 0x7fe598101930 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111950 con 0x7fe598107980 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59cdc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe598111c50 con 0x7fe5981022f0 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40154/0 (socket says 172.21.15.37:40154) 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 -- 172.21.15.37:0/3816970106 learned_addr learned my addr 172.21.15.37:0/3816970106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe595d9b700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5981022f0 0x7fe59819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe596d9d700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe598107980 0x7fe59819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 -- 172.21.15.37:0/3816970106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5981022f0 msgr2=0x7fe59819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5981022f0 0x7fe59819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 -- 172.21.15.37:0/3816970106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe598107980 msgr2=0x7fe59819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe598107980 0x7fe59819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 -- 172.21.15.37:0/3816970106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5981a0310 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.984+0000 7fe59659c700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d980 tx=0x7fe58800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.985+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588016070 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.985+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5981a0600 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.985+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe588004830 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.985+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588005540 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.985+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5981a7e80 con 0x7fe598101930 2024-04-25T21:04:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.986+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe598069730 con 0x7fe598101930 2024-04-25T21:04:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.987+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe588004220 con 0x7fe598101930 2024-04-25T21:04:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.987+0000 7fe5877fe700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 0x7fe5800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.987+0000 7fe595d9b700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 0x7fe5800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.987+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe5880d0150 con 0x7fe598101930 2024-04-25T21:04:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.988+0000 7fe595d9b700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 0x7fe5800a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0098e0 tx=0x7fe58c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:29.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:29.991+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe58801f020 con 0x7fe598101930 2024-04-25T21:04:30.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.250+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5981a8050 con 0x7fe598101930 2024-04-25T21:04:30.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.250+0000 7fe5877fe700 1 -- 172.21.15.37:0/3816970106 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5515 (secure 0 0 0) 0x7fe58809cfd0 con 0x7fe598101930 2024-04-25T21:04:30.251 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:30.251 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T21:04:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 msgr2=0x7fe5800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 0x7fe5800a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe58c0098e0 tx=0x7fe58c005960 comp rx=0 tx=0).stop 2024-04-25T21:04:30.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 msgr2=0x7fe59819b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe58800d980 tx=0x7fe58800dc90 comp rx=0 tx=0).stop 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 shutdown_connections 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe5800a3c40 0x7fe5800a6100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe598107980 0x7fe59819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe598101930 0x7fe59819b300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 --2- 172.21.15.37:0/3816970106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5981022f0 0x7fe59819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.252+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 >> 172.21.15.37:0/3816970106 conn(0x7fe5980788b0 msgr2=0x7fe59806af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.253+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 shutdown_connections 2024-04-25T21:04:30.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:30.253+0000 7fe59cdc5700 1 -- 172.21.15.37:0/3816970106 wait complete. 2024-04-25T21:04:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 podman[63420]: 2024-04-25 21:04:29.964977485 +0000 UTC m=+0.393233302 container init 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 podman[63420]: 2024-04-25 21:04:29.967863904 +0000 UTC m=+0.396119718 container start 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 ceph-mds[63439]: -- 172.21.15.177:0/1492401023 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6e5679a40 con 0x55c6e564d000 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: starting mds.c at 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:30 smithi177 bash[63420]: 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 2024-04-25T21:04:30.340 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:04:30 smithi177 systemd[1]: Started Ceph mds.c for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: Deploying daemon mds.h on smithi098 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:31.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3816970106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:31.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:boot 2024-04-25T21:04:31.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: fsmap 7 up:standby 2024-04-25T21:04:31.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:31 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:04:31.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: Deploying daemon mds.h on smithi098 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3816970106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:boot 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: fsmap 7 up:standby 2024-04-25T21:04:31.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:31 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: Deploying daemon mds.h on smithi098 2024-04-25T21:04:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3816970106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:boot 2024-04-25T21:04:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: fsmap 7 up:standby 2024-04-25T21:04:31.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:31 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:04:32.667 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:32 smithi098 systemd[1]: Starting Ceph mds.h for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:32.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:33.026 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:32 smithi098 podman[62637]: 2024-04-25 21:04:32.59461466 +0000 UTC m=+0.018003745 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:33.027 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:32 smithi098 podman[62637]: 2024-04-25 21:04:32.715868247 +0000 UTC m=+0.139257327 container create 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T21:04:33.027 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 podman[62637]: 2024-04-25 21:04:33.027052546 +0000 UTC m=+0.450441625 container init 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T21:04:33.188 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:33.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 podman[62637]: 2024-04-25 21:04:33.029595587 +0000 UTC m=+0.452984669 container start 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T21:04:33.418 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-mds[62656]: -- 172.21.15.98:0/3356405172 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db25475a40 con 0x55db25448c00 2024-04-25T21:04:33.419 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: starting mds.h at 2024-04-25T21:04:33.419 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 bash[62637]: 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 2024-04-25T21:04:33.419 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:04:33 smithi098 systemd[1]: Started Ceph mds.h for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:33.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:33.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:33 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:33.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:33.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:33 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:34.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:34 smithi037 ceph-mon[29195]: Deploying daemon mds.i on smithi177 2024-04-25T21:04:34.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:34 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:boot 2024-04-25T21:04:34.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:34 smithi037 ceph-mon[29195]: fsmap 8 up:standby 2024-04-25T21:04:34.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:04:34.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:34 smithi177 ceph-mon[33043]: Deploying daemon mds.i on smithi177 2024-04-25T21:04:34.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:34 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:boot 2024-04-25T21:04:34.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:34 smithi177 ceph-mon[33043]: fsmap 8 up:standby 2024-04-25T21:04:34.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:04:34.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:34 smithi098 ceph-mon[32569]: Deploying daemon mds.i on smithi177 2024-04-25T21:04:34.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:34 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:boot 2024-04-25T21:04:34.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:34 smithi098 ceph-mon[32569]: fsmap 8 up:standby 2024-04-25T21:04:34.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:04:34.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 -- 172.21.15.37:0/1251308479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c075590 msgr2=0x7f3f7c10ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:34.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 --2- 172.21.15.37:0/1251308479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c075590 0x7f3f7c10ef30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 -- 172.21.15.37:0/1251308479 shutdown_connections 2024-04-25T21:04:34.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 --2- 172.21.15.37:0/1251308479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 0x7f3f7c111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 --2- 172.21.15.37:0/1251308479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c074c70 0x7f3f7c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 --2- 172.21.15.37:0/1251308479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c075590 0x7f3f7c10ef30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 -- 172.21.15.37:0/1251308479 >> 172.21.15.37:0/1251308479 conn(0x7f3f7c079bf0 msgr2=0x7f3f7c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 -- 172.21.15.37:0/1251308479 shutdown_connections 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.785+0000 7f3f810a5700 1 -- 172.21.15.37:0/1251308479 wait complete. 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 Processor -- start 2024-04-25T21:04:34.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 -- start start 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c075590 0x7f3f7c19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 0x7f3f7c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c114e70 con 0x7f3f7c074c70 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c114cf0 con 0x7f3f7c10f470 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f810a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7c114ff0 con 0x7f3f7c075590 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f7ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:34.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f7ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40180/0 (socket says 172.21.15.37:40180) 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f7ad9d700 1 -- 172.21.15.37:0/275980480 learned_addr learned my addr 172.21.15.37:0/275980480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f7a59c700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c075590 0x7f3f7c19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.786+0000 7f3f7b59e700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 0x7f3f7c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 -- 172.21.15.37:0/275980480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c075590 msgr2=0x7f3f7c19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c075590 0x7f3f7c19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 -- 172.21.15.37:0/275980480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 msgr2=0x7f3f7c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 0x7f3f7c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 -- 172.21.15.37:0/275980480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7c1a0390 con 0x7f3f7c074c70 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f7ad9d700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c9b0 tx=0x7f3f6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c007b00 con 0x7f3f7c074c70 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7c1a0680 con 0x7f3f7c074c70 2024-04-25T21:04:34.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f6c00ce50 con 0x7f3f7c074c70 2024-04-25T21:04:34.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c018660 con 0x7f3f7c074c70 2024-04-25T21:04:34.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.787+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7c1a7f00 con 0x7f3f7c074c70 2024-04-25T21:04:34.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.788+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7c069730 con 0x7f3f7c074c70 2024-04-25T21:04:34.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.790+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3f6c00f450 con 0x7f3f7c074c70 2024-04-25T21:04:34.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.790+0000 7f3f63fff700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 0x7f3f640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:34.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.791+0000 7f3f7a59c700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 0x7f3f640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:34.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.791+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f3f6c0d0190 con 0x7f3f7c074c70 2024-04-25T21:04:34.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.791+0000 7f3f7a59c700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 0x7f3f640a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f700098e0 tx=0x7f3f70005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:34.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:34.794+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f6c014030 con 0x7f3f7c074c70 2024-04-25T21:04:35.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.051+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f7c19c6e0 con 0x7f3f7c074c70 2024-04-25T21:04:35.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.051+0000 7f3f63fff700 1 -- 172.21.15.37:0/275980480 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7168 (secure 0 0 0) 0x7f3f6c09d000 con 0x7f3f7c074c70 2024-04-25T21:04:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:35.052 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 msgr2=0x7f3f640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 0x7f3f640a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f700098e0 tx=0x7f3f70005960 comp rx=0 tx=0).stop 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 msgr2=0x7f3f7c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00c9b0 tx=0x7f3f6c00ccc0 comp rx=0 tx=0).stop 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 shutdown_connections 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f3f640a3c40 0x7f3f640a6100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:35.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f7c10f470 0x7f3f7c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f7c074c70 0x7f3f7c19b380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 --2- 172.21.15.37:0/275980480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f7c075590 0x7f3f7c19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.053+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 >> 172.21.15.37:0/275980480 conn(0x7f3f7c079bf0 msgr2=0x7f3f7c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.054+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 shutdown_connections 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:35.054+0000 7f3f810a5700 1 -- 172.21.15.37:0/275980480 wait complete. 2024-04-25T21:04:35.055 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T21:04:35.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:35 smithi037 ceph-mon[29195]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:35.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:35 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/275980480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:35.504 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:35 smithi177 ceph-mon[33043]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:35.505 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:35 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/275980480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:35.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:35 smithi098 ceph-mon[32569]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:35.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:35 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/275980480' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:35.840 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:35 smithi177 podman[63723]: 2024-04-25 21:04:35.735199078 +0000 UTC m=+0.128165132 container create 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:04:36.121 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 podman[63723]: 2024-04-25 21:04:36.021750746 +0000 UTC m=+0.414716801 container init 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:04:36.121 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 podman[63723]: 2024-04-25 21:04:36.02417441 +0000 UTC m=+0.417140468 container start 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:04:36.121 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 ceph-mds[63742]: -- 172.21.15.177:0/1025599656 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d499f9a40 con 0x557d499ccc00 2024-04-25T21:04:36.121 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: starting mds.i at 2024-04-25T21:04:36.121 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 bash[63723]: 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 2024-04-25T21:04:36.589 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:04:36 smithi177 systemd[1]: Started Ceph mds.i for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:37.444 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.444 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: Deploying daemon mds.g on smithi037 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:boot 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: fsmap 9 up:standby 2024-04-25T21:04:37.445 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:37 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: Deploying daemon mds.g on smithi037 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:boot 2024-04-25T21:04:37.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: fsmap 9 up:standby 2024-04-25T21:04:37.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:37 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:04:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: Deploying daemon mds.g on smithi037 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:boot 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: fsmap 9 up:standby 2024-04-25T21:04:37.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:37 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:04:37.719 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:37.980 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:39.462 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:39 smithi037 ceph-mon[29195]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:39.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:39 smithi177 ceph-mon[33043]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:39.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:39 smithi098 ceph-mon[32569]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:39.726 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:39 smithi037 podman[71785]: 2024-04-25 21:04:39.61035229 +0000 UTC m=+0.249521022 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:39.726 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:39 smithi037 podman[71785]: 2024-04-25 21:04:39.726284143 +0000 UTC m=+0.365452850 container create 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:04:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 -- 172.21.15.37:0/1047483412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40406b0f0 msgr2=0x7fb40410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/1047483412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40406b0f0 0x7fb40410f900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00b0a0 tx=0x7fb3fc00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 -- 172.21.15.37:0/1047483412 shutdown_connections 2024-04-25T21:04:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/1047483412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb40406b0f0 0x7fb40410f900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/1047483412 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb404108a30 0x7fb404108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/1047483412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406a750 0x7fb40406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 -- 172.21.15.37:0/1047483412 >> 172.21.15.37:0/1047483412 conn(0x7fb4040788f0 msgr2=0x7fb404078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 -- 172.21.15.37:0/1047483412 shutdown_connections 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.082+0000 7fb40bb5a700 1 -- 172.21.15.37:0/1047483412 wait complete. 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 Processor -- start 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 -- start start 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb40406a750 0x7fb404199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406b0f0 0x7fb404199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112cb0 con 0x7fb404108a30 2024-04-25T21:04:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112b30 con 0x7fb40406b0f0 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.083+0000 7fb40bb5a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb404112e30 con 0x7fb40406a750 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50874/0 (socket says 172.21.15.37:50874) 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 -- 172.21.15.37:0/3236262355 learned_addr learned my addr 172.21.15.37:0/3236262355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb4098f6700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb40406a750 0x7fb404199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb4090f5700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406b0f0 0x7fb404199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 -- 172.21.15.37:0/3236262355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb40406a750 msgr2=0x7fb404199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb40406a750 0x7fb404199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 -- 172.21.15.37:0/3236262355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406b0f0 msgr2=0x7fb404199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406b0f0 0x7fb404199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 -- 172.21.15.37:0/3236262355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40419e250 con 0x7fb404108a30 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40a0f7700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0062a0 tx=0x7fb3fc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:40.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc021070 con 0x7fb404108a30 2024-04-25T21:04:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40419e4e0 con 0x7fb404108a30 2024-04-25T21:04:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc007c80 con 0x7fb404108a30 2024-04-25T21:04:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc01d450 con 0x7fb404108a30 2024-04-25T21:04:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.084+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a5dc0 con 0x7fb404108a30 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.085+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40410b030 con 0x7fb404108a30 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.086+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3fc004000 con 0x7fb404108a30 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.087+0000 7fb3faffd700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.087+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb3fc0d10f0 con 0x7fb404108a30 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.087+0000 7fb4098f6700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.087+0000 7fb4098f6700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 0x7fb3f00a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb400009a10 tx=0x7fb400009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:40.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.091+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3fc09e000 con 0x7fb404108a30 2024-04-25T21:04:40.235 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 podman[71785]: 2024-04-25 21:04:40.213593047 +0000 UTC m=+0.852761754 container init 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T21:04:40.235 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 podman[71785]: 2024-04-25 21:04:40.216409644 +0000 UTC m=+0.855578349 container start 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:04:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.362+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb404069730 con 0x7fb404108a30 2024-04-25T21:04:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.363+0000 7fb3faffd700 1 -- 172.21.15.37:0/3236262355 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7998 (secure 0 0 0) 0x7fb3fc09d900 con 0x7fb404108a30 2024-04-25T21:04:40.363 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:40.364 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T21:04:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 msgr2=0x7fb3f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 0x7fb3f00a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb400009a10 tx=0x7fb400009190 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 msgr2=0x7fb40419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0062a0 tx=0x7fb3fc004e20 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 shutdown_connections 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb404108a30 0x7fb40419db10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb40406a750 0x7fb404199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb3f00a3c40 0x7fb3f00a6100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 --2- 172.21.15.37:0/3236262355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb40406b0f0 0x7fb404199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 >> 172.21.15.37:0/3236262355 conn(0x7fb4040788f0 msgr2=0x7fb4040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 shutdown_connections 2024-04-25T21:04:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:40.365+0000 7fb40bb5a700 1 -- 172.21.15.37:0/3236262355 wait complete. 2024-04-25T21:04:40.367 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-04-25T21:04:40.735 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 ceph-mds[71834]: -- 172.21.15.37:0/4072772516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c4c989a40 con 0x556c4c95c800 2024-04-25T21:04:40.735 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: starting mds.g at 2024-04-25T21:04:40.735 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 bash[71785]: 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d 2024-04-25T21:04:40.735 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:04:40 smithi037 systemd[1]: Started Ceph mds.g for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:41.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3236262355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:41.423 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:41 smithi037 ceph-mon[29195]: Deploying daemon mds.a on smithi037 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3236262355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:41.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:41 smithi177 ceph-mon[33043]: Deploying daemon mds.a on smithi037 2024-04-25T21:04:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3236262355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:41.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:41 smithi098 ceph-mon[32569]: Deploying daemon mds.a on smithi037 2024-04-25T21:04:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:42 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6838/2484283883,v1:172.21.15.37:6839/2484283883] up:boot 2024-04-25T21:04:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:42 smithi177 ceph-mon[33043]: fsmap 10 up:standby 2024-04-25T21:04:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:42 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:04:42.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:42 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6838/2484283883,v1:172.21.15.37:6839/2484283883] up:boot 2024-04-25T21:04:42.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:42 smithi037 ceph-mon[29195]: fsmap 10 up:standby 2024-04-25T21:04:42.603 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:42 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:04:42.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:42 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6838/2484283883,v1:172.21.15.37:6839/2484283883] up:boot 2024-04-25T21:04:42.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:42 smithi098 ceph-mon[32569]: fsmap 10 up:standby 2024-04-25T21:04:42.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:42 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:04:42.985 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:42 smithi037 systemd[1]: Starting Ceph mds.a for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:04:43.087 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:43.265 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:43 smithi037 ceph-mon[29195]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:43.265 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 podman[72146]: 2024-04-25 21:04:42.918503867 +0000 UTC m=+0.019141244 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:04:43.266 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 podman[72146]: 2024-04-25 21:04:43.029636615 +0000 UTC m=+0.130273991 container create d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T21:04:43.266 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 podman[72146]: 2024-04-25 21:04:43.266085678 +0000 UTC m=+0.366723057 container init d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:04:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:43.577 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 podman[72146]: 2024-04-25 21:04:43.269357495 +0000 UTC m=+0.369994880 container start d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T21:04:43.577 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 ceph-mds[72194]: -- 172.21.15.37:0/797729742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5589fb655a40 con 0x5589fb629000 2024-04-25T21:04:43.577 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: starting mds.a at 2024-04-25T21:04:43.577 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 bash[72146]: d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 2024-04-25T21:04:43.577 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:04:43 smithi037 systemd[1]: Started Ceph mds.a for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:43.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:43 smithi177 ceph-mon[33043]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:43 smithi098 ceph-mon[32569]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:44.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: Deploying daemon mds.b on smithi098 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6840/2041063205,v1:172.21.15.37:6841/2041063205] up:boot 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: fsmap 11 up:standby 2024-04-25T21:04:44.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:44 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: Deploying daemon mds.b on smithi098 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6840/2041063205,v1:172.21.15.37:6841/2041063205] up:boot 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: fsmap 11 up:standby 2024-04-25T21:04:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:44 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: Deploying daemon mds.b on smithi098 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6840/2041063205,v1:172.21.15.37:6841/2041063205] up:boot 2024-04-25T21:04:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: fsmap 11 up:standby 2024-04-25T21:04:44.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:44 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 -- 172.21.15.37:0/2033307547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 msgr2=0x7f2a8c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/2033307547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c10f930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400b0a0 tx=0x7f2a8400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 -- 172.21.15.37:0/2033307547 shutdown_connections 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/2033307547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c10f930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/2033307547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c108a60 0x7f2a8c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/2033307547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c06b1f0 0x7f2a8c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 -- 172.21.15.37:0/2033307547 >> 172.21.15.37:0/2033307547 conn(0x7f2a8c0789e0 msgr2=0x7f2a8c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 -- 172.21.15.37:0/2033307547 shutdown_connections 2024-04-25T21:04:45.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.147+0000 7f2a93a7d700 1 -- 172.21.15.37:0/2033307547 wait complete. 2024-04-25T21:04:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 Processor -- start 2024-04-25T21:04:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 -- start start 2024-04-25T21:04:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c06b1f0 0x7f2a8c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:45.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c108a60 0x7f2a8c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112c60 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112ae0 con 0x7f2a8c06b1f0 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a93a7d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112de0 con 0x7f2a8c108a60 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50920/0 (socket says 172.21.15.37:50920) 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 -- 172.21.15.37:0/3423203092 learned_addr learned my addr 172.21.15.37:0/3423203092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91819700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c06b1f0 0x7f2a8c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a9201a700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c108a60 0x7f2a8c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 -- 172.21.15.37:0/3423203092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c108a60 msgr2=0x7f2a8c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c108a60 0x7f2a8c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 -- 172.21.15.37:0/3423203092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c06b1f0 msgr2=0x7f2a8c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:45.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c06b1f0 0x7f2a8c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.148+0000 7f2a91018700 1 -- 172.21.15.37:0/3423203092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c1a0340 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a91018700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d980 tx=0x7f2a7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c017070 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c005020 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c1a0630 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c009b50 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.149+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a7eb0 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.150+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c069730 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.151+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a7c0051d0 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.152+0000 7f2a82ffd700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 0x7f2a780a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.152+0000 7f2a91819700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 0x7f2a780a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.152+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2a7c0d01e0 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.152+0000 7f2a91819700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 0x7f2a780a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a88007b60 tx=0x7f2a880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:45.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.156+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a7c020020 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.408+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8c19c690 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.409+0000 7f2a82ffd700 1 -- 172.21.15.37:0/3423203092 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9650 (secure 0 0 0) 0x7f2a7c09d0e0 con 0x7f2a8c06bbb0 2024-04-25T21:04:45.410 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:45.410 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T21:04:45.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.411+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 msgr2=0x7f2a780a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:45.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.411+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 0x7f2a780a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a88007b60 tx=0x7f2a880073d0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.411+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 msgr2=0x7f2a8c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:45.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.411+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d980 tx=0x7f2a7c00dc90 comp rx=0 tx=0).stop 2024-04-25T21:04:45.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 shutdown_connections 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a8c108a60 0x7f2a8c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2a780a3cc0 0x7f2a780a6180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a8c06b1f0 0x7f2a8c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 --2- 172.21.15.37:0/3423203092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a8c06bbb0 0x7f2a8c19b870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 >> 172.21.15.37:0/3423203092 conn(0x7f2a8c0789e0 msgr2=0x7f2a8c10ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 shutdown_connections 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:45.412+0000 7f2a93a7d700 1 -- 172.21.15.37:0/3423203092 wait complete. 2024-04-25T21:04:45.413 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-25T21:04:45.706 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:45 smithi098 ceph-mon[32569]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:45.706 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3423203092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:45.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:45 smithi037 ceph-mon[29195]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:45.732 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3423203092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:45.741 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:45 smithi177 ceph-mon[33043]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:45.741 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3423203092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:46.332 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:04:46 smithi098 systemd[1]: Started Ceph mds.b for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:04:47.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:47.615 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:boot 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: fsmap 12 up:standby 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:04:47.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:47 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:boot 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: fsmap 12 up:standby 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:04:47.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:47 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:47.727 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:boot 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:04:47.728 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:47 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:48.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:04:48.353 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:49.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:49 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:49.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:49.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:49 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:49.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:49.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:49.669 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:49 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 -- 172.21.15.37:0/2536153843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc106990 msgr2=0x7f29bc106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 --2- 172.21.15.37:0/2536153843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc106990 0x7f29bc106d70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009990 tx=0x7f29ac009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 -- 172.21.15.37:0/2536153843 shutdown_connections 2024-04-25T21:04:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 --2- 172.21.15.37:0/2536153843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc1026e0 0x7f29bc10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 --2- 172.21.15.37:0/2536153843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc106990 0x7f29bc106d70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 --2- 172.21.15.37:0/2536153843 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc101d20 0x7f29bc1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.076+0000 7f29c3514700 1 -- 172.21.15.37:0/2536153843 >> 172.21.15.37:0/2536153843 conn(0x7f29bc0788f0 msgr2=0x7f29bc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.077+0000 7f29c3514700 1 -- 172.21.15.37:0/2536153843 shutdown_connections 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.077+0000 7f29c3514700 1 -- 172.21.15.37:0/2536153843 wait complete. 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.077+0000 7f29c3514700 1 Processor -- start 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.077+0000 7f29c3514700 1 -- start start 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc1026e0 0x7f29bc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc106990 0x7f29bc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc111bb0 con 0x7f29bc101d20 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc111a30 con 0x7f29bc106990 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc111d30 con 0x7f29bc1026e0 2024-04-25T21:04:50.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42052/0 (socket says 172.21.15.37:42052) 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 -- 172.21.15.37:0/2124000129 learned_addr learned my addr 172.21.15.37:0/2124000129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c1ab1700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc106990 0x7f29bc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 -- 172.21.15.37:0/2124000129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc1026e0 msgr2=0x7f29bc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc1026e0 0x7f29bc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 -- 172.21.15.37:0/2124000129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc106990 msgr2=0x7f29bc19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc106990 0x7f29bc19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 -- 172.21.15.37:0/2124000129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc19e2b0 con 0x7f29bc101d20 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c12b0700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000c00 tx=0x7f29ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:50.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac027070 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc19e540 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ac0040e0 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00f460 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.078+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a5e20 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.080+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc103080 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.081+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f29ac02a070 con 0x7f29bc101d20 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.081+0000 7f29b27fc700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 0x7f29a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.081+0000 7f29c0aaf700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 0x7f29a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:50.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.081+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f29ac0d1370 con 0x7f29bc101d20 2024-04-25T21:04:50.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.082+0000 7f29c0aaf700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 0x7f29a80a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f29bc079110 tx=0x7f29b8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:50.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.086+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29ac031070 con 0x7f29bc101d20 2024-04-25T21:04:50.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.364+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29bc19a600 con 0x7f29bc101d20 2024-04-25T21:04:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.364+0000 7f29b27fc700 1 -- 172.21.15.37:0/2124000129 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f29ac013070 con 0x7f29bc101d20 2024-04-25T21:04:50.365 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:50.365 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14568,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14574,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24401,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.366+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 msgr2=0x7f29a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.366+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 0x7f29a80a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f29bc079110 tx=0x7f29b8006d20 comp rx=0 tx=0).stop 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.366+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 msgr2=0x7f29bc1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.366+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000c00 tx=0x7f29ac0047b0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 shutdown_connections 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f29a80a3c40 0x7f29a80a6100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29bc106990 0x7f29bc19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29bc101d20 0x7f29bc1992f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 --2- 172.21.15.37:0/2124000129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc1026e0 0x7f29bc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 >> 172.21.15.37:0/2124000129 conn(0x7f29bc0788f0 msgr2=0x7f29bc06b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 shutdown_connections 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:50.367+0000 7f29c3514700 1 -- 172.21.15.37:0/2124000129 wait complete. 2024-04-25T21:04:50.368 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-04-25T21:04:50.459 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:50 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:50.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:50 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:50.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:50 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:51.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T21:04:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:51.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2124000129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:51.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:51 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:51.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2124000129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:51.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:51 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2124000129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:51 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:51.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:53.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 -- 172.21.15.37:0/2196479546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20101930 msgr2=0x7f1f20101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 --2- 172.21.15.37:0/2196479546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20101930 0x7f1f20101db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009990 tx=0x7f1f10009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 -- 172.21.15.37:0/2196479546 shutdown_connections 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 --2- 172.21.15.37:0/2196479546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f201022f0 0x7f1f2010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 --2- 172.21.15.37:0/2196479546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f20107980 0x7f1f20107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 --2- 172.21.15.37:0/2196479546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20101930 0x7f1f20101db0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 -- 172.21.15.37:0/2196479546 >> 172.21.15.37:0/2196479546 conn(0x7f1f200788b0 msgr2=0x7f1f20078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 -- 172.21.15.37:0/2196479546 shutdown_connections 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.394+0000 7f1f272a6700 1 -- 172.21.15.37:0/2196479546 wait complete. 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 Processor -- start 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 -- start start 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f20101930 0x7f1f201034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f201022f0 0x7f1f20103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:53.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111ad0 con 0x7f1f20107980 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111950 con 0x7f1f20101930 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f272a6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1f20111c50 con 0x7f1f201022f0 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42068/0 (socket says 172.21.15.37:42068) 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25042700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f20101930 0x7f1f201034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 -- 172.21.15.37:0/1157321175 learned_addr learned my addr 172.21.15.37:0/1157321175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:53.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f24841700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f201022f0 0x7f1f20103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 -- 172.21.15.37:0/1157321175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f201022f0 msgr2=0x7f1f20103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f201022f0 0x7f1f20103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 -- 172.21.15.37:0/1157321175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f20101930 msgr2=0x7f1f201034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f20101930 0x7f1f201034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.395+0000 7f1f25843700 1 -- 172.21.15.37:0/1157321175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f201a7640 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f25843700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f18007ee0 tx=0x7f1f1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800dbd0 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1800dd70 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f201a7930 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f18005990 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.396+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201a7e20 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.397+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f20069730 con 0x7f1f20107980 2024-04-25T21:04:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.398+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f1801a020 con 0x7f1f20107980 2024-04-25T21:04:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.399+0000 7f1f167fc700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 0x7f1f0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.399+0000 7f1f25042700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 0x7f1f0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.399+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1f180cf760 con 0x7f1f20107980 2024-04-25T21:04:53.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.399+0000 7f1f25042700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 0x7f1f0c0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009be0 tx=0x7f1f1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:53.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.404+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f18010070 con 0x7f1f20107980 2024-04-25T21:04:53.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:53 smithi037 ceph-mon[29195]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:53.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:53 smithi177 ceph-mon[33043]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:53.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.634+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1f2002df80 con 0x7f1f20107980 2024-04-25T21:04:53.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.635+0000 7f1f167fc700 1 -- 172.21.15.37:0/1157321175 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f1f1809c5c0 con 0x7f1f20107980 2024-04-25T21:04:53.635 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:04:53.636 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T21:04:53.636 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:04:53.636 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T21:04:53.636 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T21:04:53.636 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:04:53.637 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T21:04:53.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 msgr2=0x7f1f0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:53.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 0x7f1f0c0a6100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009be0 tx=0x7f1f1c009260 comp rx=0 tx=0).stop 2024-04-25T21:04:53.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 msgr2=0x7f1f20103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:53.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1f18007ee0 tx=0x7f1f1800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 shutdown_connections 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f20107980 0x7f1f20103f60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f1f0c0a3c40 0x7f1f0c0a6100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f20101930 0x7f1f201034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 --2- 172.21.15.37:0/1157321175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1f201022f0 0x7f1f20103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:53.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 >> 172.21.15.37:0/1157321175 conn(0x7f1f200788b0 msgr2=0x7f1f2006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:53.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 shutdown_connections 2024-04-25T21:04:53.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:53.637+0000 7f1f272a6700 1 -- 172.21.15.37:0/1157321175 wait complete. 2024-04-25T21:04:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:53 smithi098 ceph-mon[32569]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:54.346 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T21:04:54.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T21:04:54.594 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.838 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:54 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1157321175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:54 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1157321175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T21:04:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:54.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:04:54.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:54 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1157321175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T21:04:55.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:55 smithi037 ceph-mon[29195]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:55 smithi098 ceph-mon[32569]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:55 smithi177 ceph-mon[33043]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:56.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 -- 172.21.15.37:0/3223057597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34107990 msgr2=0x7f2f34107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:56.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 --2- 172.21.15.37:0/3223057597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34107990 0x7f2f34107d70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009990 tx=0x7f2f24009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 -- 172.21.15.37:0/3223057597 shutdown_connections 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 --2- 172.21.15.37:0/3223057597 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34102300 0x7f2f3410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 --2- 172.21.15.37:0/3223057597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34107990 0x7f2f34107d70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 --2- 172.21.15.37:0/3223057597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34101940 0x7f2f34101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 -- 172.21.15.37:0/3223057597 >> 172.21.15.37:0/3223057597 conn(0x7f2f340788f0 msgr2=0x7f2f34078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 -- 172.21.15.37:0/3223057597 shutdown_connections 2024-04-25T21:04:56.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.313+0000 7f2f3993f700 1 -- 172.21.15.37:0/3223057597 wait complete. 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 Processor -- start 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 -- start start 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 0x7f2f3419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34107990 0x7f2f3419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34111b60 con 0x7f2f34101940 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f341119e0 con 0x7f2f34102300 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f3993f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f34111ce0 con 0x7f2f34107990 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f32ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42098/0 (socket says 172.21.15.37:42098) 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.314+0000 7f2f32ffd700 1 -- 172.21.15.37:0/1503785579 learned_addr learned my addr 172.21.15.37:0/1503785579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:56.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f337fe700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34107990 0x7f2f3419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f327fc700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 0x7f2f3419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 -- 172.21.15.37:0/1503785579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34107990 msgr2=0x7f2f3419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34107990 0x7f2f3419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 -- 172.21.15.37:0/1503785579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 msgr2=0x7f2f3419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 0x7f2f3419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 -- 172.21.15.37:0/1503785579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f341a0380 con 0x7f2f34101940 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f327fc700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 0x7f2f3419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f32ffd700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f240057c0 tx=0x7f2f240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24027070 con 0x7f2f34101940 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f24003f20 con 0x7f2f34101940 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f341a0610 con 0x7f2f34101940 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f341a7ef0 con 0x7f2f34101940 2024-04-25T21:04:56.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.315+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f24022bc0 con 0x7f2f34101940 2024-04-25T21:04:56.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.316+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f34069730 con 0x7f2f34101940 2024-04-25T21:04:56.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.317+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2f2402a070 con 0x7f2f34101940 2024-04-25T21:04:56.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.318+0000 7f2f3893d700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 0x7f2f1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:56.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.318+0000 7f2f327fc700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 0x7f2f1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:56.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.318+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2f240d0ed0 con 0x7f2f34101940 2024-04-25T21:04:56.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.318+0000 7f2f327fc700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 0x7f2f1c0a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2f28005fd0 tx=0x7f2f28005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:56.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.322+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f2405eff0 con 0x7f2f34101940 2024-04-25T21:04:56.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.531+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2f341a80c0 con 0x7f2f34101940 2024-04-25T21:04:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.602+0000 7f2f3893d700 1 -- 172.21.15.37:0/1503785579 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f2f24099020 con 0x7f2f34101940 2024-04-25T21:04:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 msgr2=0x7f2f1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 0x7f2f1c0a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2f28005fd0 tx=0x7f2f28005ee0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 msgr2=0x7f2f3419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2f240057c0 tx=0x7f2f240057f0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 shutdown_connections 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f34107990 0x7f2f3419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f34101940 0x7f2f3419b370 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2f1c0a3c40 0x7f2f1c0a6100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 --2- 172.21.15.37:0/1503785579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f34102300 0x7f2f3419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 >> 172.21.15.37:0/1503785579 conn(0x7f2f340788f0 msgr2=0x7f2f3406a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 shutdown_connections 2024-04-25T21:04:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:56.605+0000 7f2f3993f700 1 -- 172.21.15.37:0/1503785579 wait complete. 2024-04-25T21:04:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T21:04:56.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:56 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T21:04:56.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:56 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T21:04:57.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:56 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T21:04:57.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T21:04:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:57 smithi098 ceph-mon[32569]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:57 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T21:04:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:57 smithi098 ceph-mon[32569]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T21:04:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:04:57.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:57 smithi037 ceph-mon[29195]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:57.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:57 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T21:04:57.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:57 smithi037 ceph-mon[29195]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T21:04:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:57 smithi177 ceph-mon[33043]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:58.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:57 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1503785579' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T21:04:58.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:57 smithi177 ceph-mon[33043]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T21:04:58.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:58 smithi098 ceph-mon[32569]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T21:04:58.918 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:58 smithi037 ceph-mon[29195]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T21:04:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:58 smithi177 ceph-mon[33043]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T21:04:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 -- 172.21.15.37:0/1845932584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 msgr2=0x7f26e006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 --2- 172.21.15.37:0/1845932584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e006b440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 -- 172.21.15.37:0/1845932584 shutdown_connections 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 --2- 172.21.15.37:0/1845932584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e006b980 0x7f26e010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 --2- 172.21.15.37:0/1845932584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e0108a20 0x7f26e0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 --2- 172.21.15.37:0/1845932584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e006b440 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 -- 172.21.15.37:0/1845932584 >> 172.21.15.37:0/1845932584 conn(0x7f26e00788b0 msgr2=0x7f26e0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 -- 172.21.15.37:0/1845932584 shutdown_connections 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.612+0000 7f26e5b95700 1 -- 172.21.15.37:0/1845932584 wait complete. 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 Processor -- start 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 -- start start 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006b980 0x7f26e0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e0108a20 0x7f26e019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0112c20 con 0x7f26e006afe0 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0112aa0 con 0x7f26e0108a20 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26e5b95700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0112da0 con 0x7f26e006b980 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26df7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57706/0 (socket says 172.21.15.37:57706) 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26df7fe700 1 -- 172.21.15.37:0/3375783515 learned_addr learned my addr 172.21.15.37:0/3375783515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26deffd700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006b980 0x7f26e0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.613+0000 7f26dffff700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e0108a20 0x7f26e019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 -- 172.21.15.37:0/3375783515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006b980 msgr2=0x7f26e0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006b980 0x7f26e0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 -- 172.21.15.37:0/3375783515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e0108a20 msgr2=0x7f26e019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e0108a20 0x7f26e019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 -- 172.21.15.37:0/3375783515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e019e100 con 0x7f26e006afe0 2024-04-25T21:04:59.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26df7fe700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26d800ba50 tx=0x7f26d800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:59.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8019070 con 0x7f26e006afe0 2024-04-25T21:04:59.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e019e3f0 con 0x7f26e006afe0 2024-04-25T21:04:59.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d800cde0 con 0x7f26e006afe0 2024-04-25T21:04:59.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8007370 con 0x7f26e006afe0 2024-04-25T21:04:59.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.614+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01a5c70 con 0x7f26e006afe0 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.615+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e010b020 con 0x7f26e006afe0 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.616+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26d801c070 con 0x7f26e006afe0 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.617+0000 7f26dcff9700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 0x7f26cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.617+0000 7f26deffd700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 0x7f26cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.617+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7f26d80cfd80 con 0x7f26e006afe0 2024-04-25T21:04:59.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.617+0000 7f26deffd700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 0x7f26cc0a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b5c0 tx=0x7f26c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:04:59.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.621+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26d8023070 con 0x7f26e006afe0 2024-04-25T21:04:59.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:04:59.829+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f26e019a450 con 0x7f26e006afe0 2024-04-25T21:04:59.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:59 smithi037 ceph-mon[29195]: pgmap v295: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:59.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:59 smithi037 ceph-mon[29195]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:04:59.830 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:04:59 smithi037 ceph-mon[29195]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T21:04:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:59 smithi098 ceph-mon[32569]: pgmap v295: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:04:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:59 smithi098 ceph-mon[32569]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:04:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:04:59 smithi098 ceph-mon[32569]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T21:05:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:59 smithi177 ceph-mon[33043]: pgmap v295: 33 pgs: 2 creating+activating, 3 creating+peering, 5 active+clean, 23 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:00.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:59 smithi177 ceph-mon[33043]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:05:00.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:04:59 smithi177 ceph-mon[33043]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T21:05:00.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.631+0000 7f26dcff9700 1 -- 172.21.15.37:0/3375783515 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7f26d809c9f0 con 0x7f26e006afe0 2024-04-25T21:05:00.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.635+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 msgr2=0x7f26cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.635+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 0x7f26cc0a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b5c0 tx=0x7f26c8005960 comp rx=0 tx=0).stop 2024-04-25T21:05:00.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.635+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 msgr2=0x7f26e01990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:00.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.635+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f26d800ba50 tx=0x7f26d800bd60 comp rx=0 tx=0).stop 2024-04-25T21:05:00.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 shutdown_connections 2024-04-25T21:05:00.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f26cc0a3c40 0x7f26cc0a6100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26e0108a20 0x7f26e019d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26e006afe0 0x7f26e01990f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 --2- 172.21.15.37:0/3375783515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e006b980 0x7f26e0199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 >> 172.21.15.37:0/3375783515 conn(0x7f26e00788b0 msgr2=0x7f26e00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 shutdown_connections 2024-04-25T21:05:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:00.636+0000 7f26e5b95700 1 -- 172.21.15.37:0/3375783515 wait complete. 2024-04-25T21:05:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T21:05:00.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:00 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T21:05:00.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:00 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T21:05:01.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:00 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T21:05:01.511 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T21:05:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:01 smithi037 ceph-mon[29195]: pgmap v297: 33 pgs: 2 creating+activating, 3 creating+peering, 18 active+clean, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:01 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T21:05:01.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:01 smithi037 ceph-mon[29195]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T21:05:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:01 smithi098 ceph-mon[32569]: pgmap v297: 33 pgs: 2 creating+activating, 3 creating+peering, 18 active+clean, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:01 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T21:05:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:01 smithi098 ceph-mon[32569]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T21:05:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:01 smithi177 ceph-mon[33043]: pgmap v297: 33 pgs: 2 creating+activating, 3 creating+peering, 18 active+clean, 10 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:01 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3375783515' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T21:05:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:01 smithi177 ceph-mon[33043]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T21:05:02.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:02 smithi098 ceph-mon[32569]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T21:05:02.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:02 smithi037 ceph-mon[29195]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T21:05:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:02 smithi177 ceph-mon[33043]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/1174222549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78102a90 msgr2=0x7ffa78102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/1174222549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78102a90 0x7ffa78102f10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800b0a0 tx=0x7ffa6800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/1174222549 shutdown_connections 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/1174222549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78103450 0x7ffa7810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/1174222549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa78108a50 0x7ffa78108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/1174222549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78102a90 0x7ffa78102f10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/1174222549 >> 172.21.15.37:0/1174222549 conn(0x7ffa78079ba0 msgr2=0x7ffa78079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/1174222549 shutdown_connections 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.549+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/1174222549 wait complete. 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 Processor -- start 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 -- start start 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78102a90 0x7ffa78074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa78108a50 0x7ffa780757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78112c50 con 0x7ffa78103450 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78112ad0 con 0x7ffa78108a50 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7f0f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffa78112dd0 con 0x7ffa78102a90 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57716/0 (socket says 172.21.15.37:57716) 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 -- 172.21.15.37:0/2317272696 learned_addr learned my addr 172.21.15.37:0/2317272696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa7ce8c700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78102a90 0x7ffa78074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 -- 172.21.15.37:0/2317272696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78102a90 msgr2=0x7ffa78074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78102a90 0x7ffa78074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 -- 172.21.15.37:0/2317272696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa78108a50 msgr2=0x7ffa780757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa78108a50 0x7ffa780757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.550+0000 7ffa77fff700 1 -- 172.21.15.37:0/2317272696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa781a76d0 con 0x7ffa78103450 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa77fff700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa680062a0 tx=0x7ffa68004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68021070 con 0x7ffa78103450 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa781a7960 con 0x7ffa78103450 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa68007e30 con 0x7ffa78103450 2024-04-25T21:05:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6801d450 con 0x7ffa78103450 2024-04-25T21:05:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.551+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa781a7ef0 con 0x7ffa78103450 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.552+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa78069730 con 0x7ffa78103450 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.553+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa68004000 con 0x7ffa78103450 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.554+0000 7ffa75ffb700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 0x7ffa640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.554+0000 7ffa7ce8c700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 0x7ffa640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.554+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7ffa6800f020 con 0x7ffa78103450 2024-04-25T21:05:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.555+0000 7ffa7ce8c700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 0x7ffa640a6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:03.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.558+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa6802a020 con 0x7ffa78103450 2024-04-25T21:05:03.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:03.818+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ffa7802df80 con 0x7ffa78103450 2024-04-25T21:05:03.819 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:03 smithi037 ceph-mon[29195]: pgmap v300: 97 pgs: 2 creating+activating, 25 creating+peering, 28 active+clean, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:03.819 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:03 smithi037 ceph-mon[29195]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T21:05:03.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:03 smithi098 ceph-mon[32569]: pgmap v300: 97 pgs: 2 creating+activating, 25 creating+peering, 28 active+clean, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:03.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:03 smithi098 ceph-mon[32569]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T21:05:04.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:03 smithi177 ceph-mon[33043]: pgmap v300: 97 pgs: 2 creating+activating, 25 creating+peering, 28 active+clean, 42 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:04.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:03 smithi177 ceph-mon[33043]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T21:05:04.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.660+0000 7ffa75ffb700 1 -- 172.21.15.37:0/2317272696 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ffa6809e880 con 0x7ffa78103450 2024-04-25T21:05:04.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 msgr2=0x7ffa640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:04.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 0x7ffa640a6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005fd0 tx=0x7ffa70005db0 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 msgr2=0x7ffa78075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa680062a0 tx=0x7ffa68004f10 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 shutdown_connections 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffa640a3cc0 0x7ffa640a6180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa78108a50 0x7ffa780757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa78102a90 0x7ffa78074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 --2- 172.21.15.37:0/2317272696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa78103450 0x7ffa78075280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 >> 172.21.15.37:0/2317272696 conn(0x7ffa78079ba0 msgr2=0x7ffa781017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 shutdown_connections 2024-04-25T21:05:04.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:04.663+0000 7ffa7f0f0700 1 -- 172.21.15.37:0/2317272696 wait complete. 2024-04-25T21:05:04.665 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T21:05:04.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:04 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T21:05:04.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:04 smithi098 ceph-mon[32569]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T21:05:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:04 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T21:05:04.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:04 smithi037 ceph-mon[29195]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T21:05:05.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:04 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T21:05:05.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:04 smithi177 ceph-mon[33043]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-25T21:05:05.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:05:05.586 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: pgmap v302: 97 pgs: 22 creating+peering, 52 active+clean, 23 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:active 2024-04-25T21:05:05.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:05 smithi098 ceph-mon[32569]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:05.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: pgmap v302: 97 pgs: 22 creating+peering, 52 active+clean, 23 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:05.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T21:05:05.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T21:05:05.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T21:05:05.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T21:05:05.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:05:05.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:active 2024-04-25T21:05:05.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:05 smithi037 ceph-mon[29195]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: pgmap v302: 97 pgs: 22 creating+peering, 52 active+clean, 23 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2317272696' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] up:active 2024-04-25T21:05:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:05 smithi177 ceph-mon[33043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:06.590 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:05:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: 2024-04-25T21:05:06.262+0000 7fe7f910d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T21:05:06.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:06 smithi037 ceph-mon[29195]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T21:05:06.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:06 smithi037 ceph-mon[29195]: Cluster is now healthy 2024-04-25T21:05:07.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:06 smithi177 ceph-mon[33043]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T21:05:07.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:06 smithi177 ceph-mon[33043]: Cluster is now healthy 2024-04-25T21:05:07.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:06 smithi098 ceph-mon[32569]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T21:05:07.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:06 smithi098 ceph-mon[32569]: Cluster is now healthy 2024-04-25T21:05:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 -- 172.21.15.37:0/611508177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8102a90 msgr2=0x7f81b8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 --2- 172.21.15.37:0/611508177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8102a90 0x7f81b8102f10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009990 tx=0x7f81ac009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 -- 172.21.15.37:0/611508177 shutdown_connections 2024-04-25T21:05:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 --2- 172.21.15.37:0/611508177 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8103450 0x7f81b810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 --2- 172.21.15.37:0/611508177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8108a80 0x7f81b8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 --2- 172.21.15.37:0/611508177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8102a90 0x7f81b8102f10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 -- 172.21.15.37:0/611508177 >> 172.21.15.37:0/611508177 conn(0x7f81b8079bf0 msgr2=0x7f81b807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 -- 172.21.15.37:0/611508177 shutdown_connections 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.201+0000 7f81be03c700 1 -- 172.21.15.37:0/611508177 wait complete. 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 Processor -- start 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 -- start start 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 0x7f81b8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8108a80 0x7f81b80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8112c90 con 0x7f81b8103450 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8112b10 con 0x7f81b8108a80 2024-04-25T21:05:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81be03c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81b8112e10 con 0x7f81b8102a90 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81b6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81b6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57726/0 (socket says 172.21.15.37:57726) 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81b6ffd700 1 -- 172.21.15.37:0/2413198759 learned_addr learned my addr 172.21.15.37:0/2413198759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81b77fe700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 0x7f81b8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.202+0000 7f81b7fff700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8108a80 0x7f81b80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 -- 172.21.15.37:0/2413198759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 msgr2=0x7f81b8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 0x7f81b8074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 -- 172.21.15.37:0/2413198759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8108a80 msgr2=0x7f81b80757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8108a80 0x7f81b80757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 -- 172.21.15.37:0/2413198759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b81a76b0 con 0x7f81b8103450 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b77fe700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 0x7f81b8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b6ffd700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000c00 tx=0x7f81ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:07.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac027070 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac003fa0 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b81a7940 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac022db0 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.203+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b81a7e70 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.204+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b8069730 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.205+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81ac02a070 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.206+0000 7f81b4ff9700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 0x7f81a00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.206+0000 7f81b77fe700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 0x7f81a00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.206+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f81ac0d1370 con 0x7f81b8103450 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.206+0000 7f81b77fe700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 0x7f81a00a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a80099d0 tx=0x7f81a8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:07.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.209+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81ac013070 con 0x7f81b8103450 2024-04-25T21:05:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.417+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81b802e140 con 0x7f81b8103450 2024-04-25T21:05:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.419+0000 7f81b4ff9700 1 -- 172.21.15.37:0/2413198759 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22136 (secure 0 0 0) 0x7f81ac09de90 con 0x7f81b8103450 2024-04-25T21:05:07.420 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:07.420 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:03.820547+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:05:07.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 msgr2=0x7f81a00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 0x7f81a00a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a80099d0 tx=0x7f81a8008040 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 msgr2=0x7f81b8075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f81ac000c00 tx=0x7f81ac0047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 shutdown_connections 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f81a00a3cc0 0x7f81a00a6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b8108a80 0x7f81b80757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b8102a90 0x7f81b8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 --2- 172.21.15.37:0/2413198759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b8103450 0x7f81b8075280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 >> 172.21.15.37:0/2413198759 conn(0x7f81b8079bf0 msgr2=0x7f81b8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 shutdown_connections 2024-04-25T21:05:07.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:07.421+0000 7f81be03c700 1 -- 172.21.15.37:0/2413198759 wait complete. 2024-04-25T21:05:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:07 smithi037 ceph-mon[29195]: pgmap v304: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 920 B/s wr, 1 op/s 2024-04-25T21:05:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:07 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2413198759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:07 smithi177 ceph-mon[33043]: pgmap v304: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 920 B/s wr, 1 op/s 2024-04-25T21:05:08.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:07 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2413198759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:08.129 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:05:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:07 smithi098 ceph-mon[32569]: pgmap v304: 97 pgs: 10 creating+peering, 87 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 920 B/s wr, 1 op/s 2024-04-25T21:05:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:07 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2413198759' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:09.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:09 smithi037 ceph-mon[29195]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s wr, 5 op/s 2024-04-25T21:05:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:09 smithi177 ceph-mon[33043]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s wr, 5 op/s 2024-04-25T21:05:10.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:09 smithi098 ceph-mon[32569]: pgmap v305: 97 pgs: 97 active+clean; 578 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s wr, 5 op/s 2024-04-25T21:05:10.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/3816357156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 msgr2=0x7f2bbc06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:10.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/3816357156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc06abb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009990 tx=0x7f2bac009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/3816357156 shutdown_connections 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/3816357156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc06b0f0 0x7f2bbc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/3816357156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc108a30 0x7f2bbc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/3816357156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc06abb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/3816357156 >> 172.21.15.37:0/3816357156 conn(0x7f2bbc0788f0 msgr2=0x7f2bbc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/3816357156 shutdown_connections 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.191+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/3816357156 wait complete. 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 Processor -- start 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 -- start start 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc06b0f0 0x7f2bbc101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 0x7f2bbc102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:10.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc112c20 con 0x7f2bbc06a750 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc112aa0 con 0x7f2bbc108a30 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc3f9f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc112da0 con 0x7f2bbc06b0f0 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc1d3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc1d3b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56720/0 (socket says 172.21.15.37:56720) 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc1d3b700 1 -- 172.21.15.37:0/437839107 learned_addr learned my addr 172.21.15.37:0/437839107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.192+0000 7f2bc153a700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc06b0f0 0x7f2bbc101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc253c700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 0x7f2bbc102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 -- 172.21.15.37:0/437839107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc06b0f0 msgr2=0x7f2bbc101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc06b0f0 0x7f2bbc101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 -- 172.21.15.37:0/437839107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 msgr2=0x7f2bbc102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 0x7f2bbc102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 -- 172.21.15.37:0/437839107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc1a76c0 con 0x7f2bbc06a750 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc253c700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 0x7f2bbc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:10.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc1d3b700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d980 tx=0x7f2bb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:10.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8017070 con 0x7f2bbc06a750 2024-04-25T21:05:10.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb8005020 con 0x7f2bbc06a750 2024-04-25T21:05:10.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1a79b0 con 0x7f2bbc06a750 2024-04-25T21:05:10.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8009b50 con 0x7f2bbc06a750 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.193+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1a7e70 con 0x7f2bbc06a750 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.194+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bbc069730 con 0x7f2bbc06a750 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.196+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2bb80051d0 con 0x7f2bbc06a750 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.196+0000 7f2bb2ffd700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 0x7f2ba80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.196+0000 7f2bc153a700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 0x7f2ba80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.196+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f2bb80d0110 con 0x7f2bbc06a750 2024-04-25T21:05:10.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.197+0000 7f2bc153a700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 0x7f2ba80a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc1034a0 tx=0x7f2bac005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:10.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.201+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bb8020020 con 0x7f2bbc06a750 2024-04-25T21:05:10.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.409+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bbc02df80 con 0x7f2bbc06a750 2024-04-25T21:05:10.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.411+0000 7f2bb2ffd700 1 -- 172.21.15.37:0/437839107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22136 (secure 0 0 0) 0x7f2bb809cc40 con 0x7f2bbc06a750 2024-04-25T21:05:10.411 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:10.412 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:03.820547+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:05:10.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 msgr2=0x7f2ba80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:10.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 0x7f2ba80a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc1034a0 tx=0x7f2bac005a90 comp rx=0 tx=0).stop 2024-04-25T21:05:10.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 msgr2=0x7f2bbc101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800d980 tx=0x7f2bb800dc90 comp rx=0 tx=0).stop 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 shutdown_connections 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2ba80a3c40 0x7f2ba80a6100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bbc108a30 0x7f2bbc102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc06a750 0x7f2bbc101a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 --2- 172.21.15.37:0/437839107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bbc06b0f0 0x7f2bbc101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 >> 172.21.15.37:0/437839107 conn(0x7f2bbc0788f0 msgr2=0x7f2bbc0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 shutdown_connections 2024-04-25T21:05:10.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:10.413+0000 7f2bc3f9f700 1 -- 172.21.15.37:0/437839107 wait complete. 2024-04-25T21:05:10.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:10 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/437839107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:11.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:10 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/437839107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:11.124 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T21:05:11.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:10 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/437839107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:11.374 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:11.750 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:11 smithi037 ceph-mon[29195]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T21:05:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:11 smithi177 ceph-mon[33043]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T21:05:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:11 smithi098 ceph-mon[32569]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T21:05:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- 172.21.15.37:0/1983754256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 msgr2=0x7fe95410f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 --2- 172.21.15.37:0/1983754256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe95410f8d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b0a0 tx=0x7fe94c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- 172.21.15.37:0/1983754256 shutdown_connections 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 --2- 172.21.15.37:0/1983754256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe95410f8d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 --2- 172.21.15.37:0/1983754256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954108a70 0x7fe954108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 --2- 172.21.15.37:0/1983754256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954102ac0 0x7fe954102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- 172.21.15.37:0/1983754256 >> 172.21.15.37:0/1983754256 conn(0x7fe954079bb0 msgr2=0x7fe954079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- 172.21.15.37:0/1983754256 shutdown_connections 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- 172.21.15.37:0/1983754256 wait complete. 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 Processor -- start 2024-04-25T21:05:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.123+0000 7fe95a880700 1 -- start start 2024-04-25T21:05:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954102ac0 0x7fe954074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 0x7fe9540757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe954112c40 con 0x7fe954103480 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe954112ac0 con 0x7fe954102ac0 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe95a880700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe954112dc0 con 0x7fe954108a70 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56734/0 (socket says 172.21.15.37:56734) 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 -- 172.21.15.37:0/2538699162 learned_addr learned my addr 172.21.15.37:0/2538699162 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe953fff700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954102ac0 0x7fe954074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe958e1d700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 0x7fe9540757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 -- 172.21.15.37:0/2538699162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 msgr2=0x7fe9540757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 0x7fe9540757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 -- 172.21.15.37:0/2538699162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954102ac0 msgr2=0x7fe954074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954102ac0 0x7fe954074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 -- 172.21.15.37:0/2538699162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9541a7570 con 0x7fe954103480 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe958e1d700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 0x7fe9540757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.124+0000 7fe9537fe700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe94800d980 tx=0x7fe94800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.125+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948017070 con 0x7fe954103480 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.125+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe948005020 con 0x7fe954103480 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.125+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541a7860 con 0x7fe954103480 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.125+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948009b50 con 0x7fe954103480 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.125+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541a7d50 con 0x7fe954103480 2024-04-25T21:05:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.126+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe954069730 con 0x7fe954103480 2024-04-25T21:05:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.127+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe9480051d0 con 0x7fe954103480 2024-04-25T21:05:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.127+0000 7fe9517fa700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 0x7fe93c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.128+0000 7fe953fff700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 0x7fe93c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.128+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe9480d0560 con 0x7fe954103480 2024-04-25T21:05:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.128+0000 7fe953fff700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 0x7fe93c0a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe94400abd0 tx=0x7fe94400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.131+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe948020020 con 0x7fe954103480 2024-04-25T21:05:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.391+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe95402e0a0 con 0x7fe954103480 2024-04-25T21:05:13.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.717+0000 7fe9517fa700 1 -- 172.21.15.37:0/2538699162 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe94809d090 con 0x7fe954103480 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 msgr2=0x7fe93c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 0x7fe93c0a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe94400abd0 tx=0x7fe94400a380 comp rx=0 tx=0).stop 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 msgr2=0x7fe954075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe94800d980 tx=0x7fe94800dc90 comp rx=0 tx=0).stop 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 shutdown_connections 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe954108a70 0x7fe9540757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe93c0a3c40 0x7fe93c0a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe954102ac0 0x7fe954074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 --2- 172.21.15.37:0/2538699162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe954103480 0x7fe954075280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.719+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 >> 172.21.15.37:0/2538699162 conn(0x7fe954079bb0 msgr2=0x7fe954101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.720+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 shutdown_connections 2024-04-25T21:05:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:13.720+0000 7fe95a880700 1 -- 172.21.15.37:0/2538699162 wait complete. 2024-04-25T21:05:13.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:13 smithi037 ceph-mon[29195]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T21:05:13.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:13 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T21:05:14.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:13 smithi177 ceph-mon[33043]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T21:05:14.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:13 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T21:05:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:13 smithi098 ceph-mon[32569]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T21:05:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:13 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T21:05:14.454 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T21:05:14.455 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-04-25T21:05:14.712 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:14.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:14 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T21:05:14.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:14 smithi037 ceph-mon[29195]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:15.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:14 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T21:05:15.068 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:14 smithi098 ceph-mon[32569]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:14 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2538699162' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T21:05:15.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:14 smithi177 ceph-mon[33043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:15.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:15 smithi037 ceph-mon[29195]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T21:05:15.991 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:15 smithi177 ceph-mon[33043]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T21:05:16.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:15 smithi098 ceph-mon[32569]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 -- 172.21.15.37:0/349291407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554102ac0 msgr2=0x7f7554102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 --2- 172.21.15.37:0/349291407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554102ac0 0x7f7554102f40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7544009990 tx=0x7f7544009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 -- 172.21.15.37:0/349291407 shutdown_connections 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 --2- 172.21.15.37:0/349291407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f755410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 --2- 172.21.15.37:0/349291407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554108a70 0x7f7554108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 --2- 172.21.15.37:0/349291407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554102ac0 0x7f7554102f40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.347+0000 7f755b335700 1 -- 172.21.15.37:0/349291407 >> 172.21.15.37:0/349291407 conn(0x7f7554079bb0 msgr2=0x7f7554079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:16.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.348+0000 7f755b335700 1 -- 172.21.15.37:0/349291407 shutdown_connections 2024-04-25T21:05:16.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.348+0000 7f755b335700 1 -- 172.21.15.37:0/349291407 wait complete. 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.348+0000 7f755b335700 1 Processor -- start 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.348+0000 7f755b335700 1 -- start start 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.348+0000 7f755b335700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554102ac0 0x7f7554074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f7554075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7554112c00 con 0x7f7554108a70 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7554112a80 con 0x7f7554103480 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7554112d80 con 0x7f7554102ac0 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56758/0 (socket says 172.21.15.37:56758) 2024-04-25T21:05:16.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 -- 172.21.15.37:0/3323696899 learned_addr learned my addr 172.21.15.37:0/3323696899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75588d0700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f7554075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75590d1700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554102ac0 0x7f7554074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 -- 172.21.15.37:0/3323696899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554102ac0 msgr2=0x7f7554074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554102ac0 0x7f7554074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 -- 172.21.15.37:0/3323696899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 msgr2=0x7f7554075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f7554075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 -- 172.21.15.37:0/3323696899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75541a75a0 con 0x7f7554108a70 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75588d0700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f7554075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f75598d2700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ef30 tx=0x7f754c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c0092e0 con 0x7f7554108a70 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75541a7830 con 0x7f7554108a70 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.349+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f754c004bb0 con 0x7f7554108a70 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.350+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754c004280 con 0x7f7554108a70 2024-04-25T21:05:16.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.350+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75541a7d20 con 0x7f7554108a70 2024-04-25T21:05:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.350+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7554069730 con 0x7f7554108a70 2024-04-25T21:05:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.352+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f754c01b030 con 0x7f7554108a70 2024-04-25T21:05:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.352+0000 7f754a7fc700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:16.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.352+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f754c0cff60 con 0x7f7554108a70 2024-04-25T21:05:16.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.352+0000 7f75590d1700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:16.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.353+0000 7f75590d1700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 0x7f75400a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7550005fd0 tx=0x7f755000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:16.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.357+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f754c022020 con 0x7f7554108a70 2024-04-25T21:05:16.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.615+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f755402df80 con 0x7f7554108a70 2024-04-25T21:05:16.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.734+0000 7f754a7fc700 1 -- 172.21.15.37:0/3323696899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f754c010040 con 0x7f7554108a70 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 msgr2=0x7f75400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 0x7f75400a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7550005fd0 tx=0x7f755000af20 comp rx=0 tx=0).stop 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 msgr2=0x7f75540757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f754c00ef30 tx=0x7f754c00c450 comp rx=0 tx=0).stop 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 shutdown_connections 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7554108a70 0x7f75540757c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554102ac0 0x7f7554074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 --2- 172.21.15.37:0/3323696899 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7554103480 0x7f7554075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 >> 172.21.15.37:0/3323696899 conn(0x7f7554079bb0 msgr2=0x7f75541016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 shutdown_connections 2024-04-25T21:05:16.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:16.736+0000 7f755b335700 1 -- 172.21.15.37:0/3323696899 wait complete. 2024-04-25T21:05:16.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:16 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T21:05:17.007 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:16 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T21:05:17.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:16 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T21:05:17.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T21:05:17.741 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T21:05:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:17 smithi037 ceph-mon[29195]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T21:05:18.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:17 smithi177 ceph-mon[33043]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T21:05:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: pgmap v309: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T21:05:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T21:05:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3323696899' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T21:05:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T21:05:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-25T21:05:18.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-04-25T21:05:18.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:17 smithi098 ceph-mon[32569]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-04-25T21:05:18.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:18 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:active 2024-04-25T21:05:18.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:18 smithi037 ceph-mon[29195]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T21:05:18.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:18 smithi037 ceph-mon[29195]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T21:05:18.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:18 smithi037 ceph-mon[29195]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T21:05:18.867 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:18 smithi037 ceph-mon[29195]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T21:05:19.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:18 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:active 2024-04-25T21:05:19.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:18 smithi177 ceph-mon[33043]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T21:05:19.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:18 smithi177 ceph-mon[33043]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T21:05:19.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:18 smithi177 ceph-mon[33043]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T21:05:19.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:18 smithi177 ceph-mon[33043]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T21:05:19.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:18 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] up:active 2024-04-25T21:05:19.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:18 smithi098 ceph-mon[32569]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T21:05:19.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:18 smithi098 ceph-mon[32569]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-04-25T21:05:19.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:18 smithi098 ceph-mon[32569]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-04-25T21:05:19.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:18 smithi098 ceph-mon[32569]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T21:05:19.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- 172.21.15.37:0/1908790636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06bbb0 msgr2=0x7f745c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 --2- 172.21.15.37:0/1908790636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06bbb0 0x7f745c10f930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f745400b0a0 tx=0x7f745400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- 172.21.15.37:0/1908790636 shutdown_connections 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 --2- 172.21.15.37:0/1908790636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06bbb0 0x7f745c10f930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 --2- 172.21.15.37:0/1908790636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c108a60 0x7f745c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 --2- 172.21.15.37:0/1908790636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c06b1f0 0x7f745c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- 172.21.15.37:0/1908790636 >> 172.21.15.37:0/1908790636 conn(0x7f745c0789e0 msgr2=0x7f745c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- 172.21.15.37:0/1908790636 shutdown_connections 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- 172.21.15.37:0/1908790636 wait complete. 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 Processor -- start 2024-04-25T21:05:19.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.395+0000 7f7462602700 1 -- start start 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c06bbb0 0x7f745c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 0x7f745c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f745c112c40 con 0x7f745c06b1f0 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f745c112ac0 con 0x7f745c06bbb0 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7462602700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f745c112dc0 con 0x7f745c108a60 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59730/0 (socket says 172.21.15.37:59730) 2024-04-25T21:05:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 -- 172.21.15.37:0/2589830593 learned_addr learned my addr 172.21.15.37:0/2589830593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745b7fe700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c06bbb0 0x7f745c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7460b9f700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 0x7f745c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 -- 172.21.15.37:0/2589830593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 msgr2=0x7f745c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 0x7f745c19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 -- 172.21.15.37:0/2589830593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c06bbb0 msgr2=0x7f745c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c06bbb0 0x7f745c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 -- 172.21.15.37:0/2589830593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c19e0b0 con 0x7f745c06b1f0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f7460b9f700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 0x7f745c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f745bfff700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c920 tx=0x7f744c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.396+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c004a30 con 0x7f745c06b1f0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.397+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744c0054d0 con 0x7f745c06b1f0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.397+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c19e3a0 con 0x7f745c06b1f0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.397+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c007c70 con 0x7f745c06b1f0 2024-04-25T21:05:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.397+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c1a5c20 con 0x7f745c06b1f0 2024-04-25T21:05:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.398+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c069730 con 0x7f745c06b1f0 2024-04-25T21:05:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.399+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f744c004c60 con 0x7f745c06b1f0 2024-04-25T21:05:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.399+0000 7f74597fa700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 0x7f74480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.399+0000 7f745b7fe700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 0x7f74480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.400+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f744c0d00d0 con 0x7f745c06b1f0 2024-04-25T21:05:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.400+0000 7f745b7fe700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 0x7f74480a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f745c078f50 tx=0x7f7444009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.403+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f744c014030 con 0x7f745c06b1f0 2024-04-25T21:05:19.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.658+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f745c02df80 con 0x7f745c06b1f0 2024-04-25T21:05:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.759+0000 7f74597fa700 1 -- 172.21.15.37:0/2589830593 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f744c09cc00 con 0x7f745c06b1f0 2024-04-25T21:05:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.761+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 msgr2=0x7f74480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.761+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 0x7f74480a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f745c078f50 tx=0x7f7444009260 comp rx=0 tx=0).stop 2024-04-25T21:05:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 msgr2=0x7f745c1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f744c00c920 tx=0x7f744c00cc30 comp rx=0 tx=0).stop 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 shutdown_connections 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f745c108a60 0x7f745c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f745c06b1f0 0x7f745c1990f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f74480a3c40 0x7f74480a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 --2- 172.21.15.37:0/2589830593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f745c06bbb0 0x7f745c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 >> 172.21.15.37:0/2589830593 conn(0x7f745c0789e0 msgr2=0x7f745c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 shutdown_connections 2024-04-25T21:05:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:19.762+0000 7f7462602700 1 -- 172.21.15.37:0/2589830593 wait complete. 2024-04-25T21:05:19.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 3 op/s 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:active 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T21:05:20.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:19 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T21:05:20.093 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 3 op/s 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:active 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T21:05:20.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:19 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T21:05:20.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: pgmap v310: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 3 op/s 2024-04-25T21:05:20.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] up:active 2024-04-25T21:05:20.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T21:05:20.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2024-04-25T21:05:20.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-04-25T21:05:20.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-04-25T21:05:20.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:19 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T21:05:20.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T21:05:20.782 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:21.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:active 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: Cluster is now healthy 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:active 2024-04-25T21:05:21.055 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:20 smithi037 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:21.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:active 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: Cluster is now healthy 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:active 2024-04-25T21:05:21.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:20 smithi177 ceph-mon[33043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:21.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2589830593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T21:05:21.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] up:active 2024-04-25T21:05:21.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T21:05:21.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T21:05:21.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: Cluster is now healthy 2024-04-25T21:05:21.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-04-25T21:05:21.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T21:05:21.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-04-25T21:05:21.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] up:active 2024-04-25T21:05:21.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:20 smithi098 ceph-mon[32569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:21.590 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:05:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: 2024-04-25T21:05:21.263+0000 7fe7f910d700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T21:05:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:21 smithi037 ceph-mon[29195]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T21:05:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:21 smithi177 ceph-mon[33043]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T21:05:22.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:21 smithi098 ceph-mon[32569]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T21:05:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1247574838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006b1f0 msgr2=0x7fd6d006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1247574838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006b1f0 0x7fd6d006b670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009990 tx=0x7fd6c0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1247574838 shutdown_connections 2024-04-25T21:05:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1247574838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006bbb0 0x7fd6d010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1247574838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1247574838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006b1f0 0x7fd6d006b670 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.636+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1247574838 >> 172.21.15.37:0/1247574838 conn(0x7fd6d00789e0 msgr2=0x7fd6d0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1247574838 shutdown_connections 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1247574838 wait complete. 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 Processor -- start 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- start start 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 0x7fd6d01019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112ca0 con 0x7fd6d006bbb0 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112b20 con 0x7fd6d006b1f0 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d79a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112e20 con 0x7fd6d0108a60 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.637+0000 7fd6d4f40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59748/0 (socket says 172.21.15.37:59748) 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 -- 172.21.15.37:0/1398827422 learned_addr learned my addr 172.21.15.37:0/1398827422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d5741700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 0x7fd6d01019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d5f42700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 -- 172.21.15.37:0/1398827422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 msgr2=0x7fd6d0102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 -- 172.21.15.37:0/1398827422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 msgr2=0x7fd6d01019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 0x7fd6d01019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 -- 172.21.15.37:0/1398827422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a7610 con 0x7fd6d006bbb0 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d5f42700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d5741700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 0x7fd6d01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d4f40700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0027070 con 0x7fd6d006bbb0 2024-04-25T21:05:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a78a0 con 0x7fd6d006bbb0 2024-04-25T21:05:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c00041d0 con 0x7fd6d006bbb0 2024-04-25T21:05:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c000f460 con 0x7fd6d006bbb0 2024-04-25T21:05:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.638+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a7d50 con 0x7fd6d006bbb0 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.639+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d010aff0 con 0x7fd6d006bbb0 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.641+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd6c002a070 con 0x7fd6d006bbb0 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.641+0000 7fd6c67fc700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 0x7fd6bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.641+0000 7fd6d5741700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 0x7fd6bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.641+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd6c00d16f0 con 0x7fd6d006bbb0 2024-04-25T21:05:22.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.642+0000 7fd6d5741700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 0x7fd6bc0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.645+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6c0031070 con 0x7fd6d006bbb0 2024-04-25T21:05:22.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:22.902+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd6d0102e90 con 0x7fd6d006bbb0 2024-04-25T21:05:23.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.775+0000 7fd6c67fc700 1 -- 172.21.15.37:0/1398827422 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fd6c0013070 con 0x7fd6d006bbb0 2024-04-25T21:05:23.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 msgr2=0x7fd6bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 0x7fd6bc0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc005fd0 tx=0x7fd6cc005d90 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 msgr2=0x7fd6d0101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0000c00 tx=0x7fd6c00047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 shutdown_connections 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0108a60 0x7fd6d0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd6bc0a3c40 0x7fd6bc0a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d006b1f0 0x7fd6d01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 --2- 172.21.15.37:0/1398827422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d006bbb0 0x7fd6d0101ef0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 >> 172.21.15.37:0/1398827422 conn(0x7fd6d00789e0 msgr2=0x7fd6d00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:23.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 shutdown_connections 2024-04-25T21:05:23.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:23.778+0000 7fd6d79a5700 1 -- 172.21.15.37:0/1398827422 wait complete. 2024-04-25T21:05:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:23 smithi177 ceph-mon[33043]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T21:05:24.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:23 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T21:05:24.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:23 smithi037 ceph-mon[29195]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T21:05:24.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:23 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T21:05:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:23 smithi098 ceph-mon[32569]: pgmap v312: 97 pgs: 97 active+clean; 583 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-04-25T21:05:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:23 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T21:05:24.560 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T21:05:24.819 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:25.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:24 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T21:05:25.053 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:24 smithi037 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:25.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:24 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T21:05:25.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:24 smithi177 ceph-mon[33043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:25.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:24 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1398827422' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T21:05:25.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:24 smithi098 ceph-mon[32569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:05:25.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:25 smithi037 ceph-mon[29195]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-04-25T21:05:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:25 smithi177 ceph-mon[33043]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-04-25T21:05:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:25 smithi098 ceph-mon[32569]: pgmap v313: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.0 KiB/s wr, 15 op/s 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 -- 172.21.15.37:0/498007388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8108a70 msgr2=0x7fefa8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 --2- 172.21.15.37:0/498007388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8108a70 0x7fefa8108e50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009990 tx=0x7fefa4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 -- 172.21.15.37:0/498007388 shutdown_connections 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 --2- 172.21.15.37:0/498007388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 0x7fefa810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 --2- 172.21.15.37:0/498007388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8108a70 0x7fefa8108e50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 --2- 172.21.15.37:0/498007388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8102ac0 0x7fefa8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 -- 172.21.15.37:0/498007388 >> 172.21.15.37:0/498007388 conn(0x7fefa8079bb0 msgr2=0x7fefa8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 -- 172.21.15.37:0/498007388 shutdown_connections 2024-04-25T21:05:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.597+0000 7fefb021e700 1 -- 172.21.15.37:0/498007388 wait complete. 2024-04-25T21:05:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 Processor -- start 2024-04-25T21:05:26.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 -- start start 2024-04-25T21:05:26.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 0x7fefa8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8108a70 0x7fefa819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112d50 con 0x7fefa8102ac0 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112bd0 con 0x7fefa8103480 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefb021e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112ed0 con 0x7fefa8108a70 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefadfba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefadfba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59768/0 (socket says 172.21.15.37:59768) 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefadfba700 1 -- 172.21.15.37:0/187304198 learned_addr learned my addr 172.21.15.37:0/187304198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefad7b9700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 0x7fefa8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.598+0000 7fefae7bb700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8108a70 0x7fefa819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 -- 172.21.15.37:0/187304198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8108a70 msgr2=0x7fefa819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8108a70 0x7fefa819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 -- 172.21.15.37:0/187304198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 msgr2=0x7fefa8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 0x7fefa8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 -- 172.21.15.37:0/187304198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa819e2f0 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefadfba700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005910 tx=0x7fefa40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4027070 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa4003fa0 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa819e580 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4022d50 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.599+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a5e60 con 0x7fefa8102ac0 2024-04-25T21:05:26.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.600+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8069730 con 0x7fefa8102ac0 2024-04-25T21:05:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.601+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fefa402a070 con 0x7fefa8102ac0 2024-04-25T21:05:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.602+0000 7fef9effd700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 0x7fef940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:26.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.602+0000 7fefad7b9700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 0x7fef940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.602+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fefa40d11d0 con 0x7fefa8102ac0 2024-04-25T21:05:26.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.602+0000 7fefad7b9700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 0x7fef940a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fefa807a2f0 tx=0x7fef98009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:26.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.607+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefa405eff0 con 0x7fefa8102ac0 2024-04-25T21:05:26.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.845+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fefa819a640 con 0x7fef940a3c40 2024-04-25T21:05:26.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.873+0000 7fef9effd700 1 -- 172.21.15.37:0/187304198 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fefa819a640 con 0x7fef940a3c40 2024-04-25T21:05:26.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 msgr2=0x7fef940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 0x7fef940a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fefa807a2f0 tx=0x7fef98009190 comp rx=0 tx=0).stop 2024-04-25T21:05:26.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 msgr2=0x7fefa81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005910 tx=0x7fefa40047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 shutdown_connections 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefa8108a70 0x7fefa819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefa8102ac0 0x7fefa81992e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fef940a3c40 0x7fef940a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 --2- 172.21.15.37:0/187304198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefa8103480 0x7fefa8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 >> 172.21.15.37:0/187304198 conn(0x7fefa8079bb0 msgr2=0x7fefa81019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 shutdown_connections 2024-04-25T21:05:26.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:26.875+0000 7fefb021e700 1 -- 172.21.15.37:0/187304198 wait complete. 2024-04-25T21:05:27.225 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:05:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:05:26.850+0000 7f91c4740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:05:27.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:05:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:05:26.850+0000 7f91c4740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:05:27.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:05:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:05:26.850+0000 7f91c4740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:05:27.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:05:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:05:26.850+0000 7f91c4740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:05:27.226 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:05:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:05:26.850+0000 7f91c4740700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:05:27.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T21:05:27.830 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:27.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:27 smithi037 ceph-mon[29195]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T21:05:27.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:27 smithi037 ceph-mon[29195]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:27 smithi177 ceph-mon[33043]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T21:05:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:27 smithi177 ceph-mon[33043]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:27 smithi098 ceph-mon[32569]: pgmap v314: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-04-25T21:05:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:27 smithi098 ceph-mon[32569]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:29.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 -- 172.21.15.37:0/1526271536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 msgr2=0x7f7be406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/1526271536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be406b670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 -- 172.21.15.37:0/1526271536 shutdown_connections 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/1526271536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be406bbb0 0x7f7be410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/1526271536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be4108a60 0x7f7be4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/1526271536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be406b670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 -- 172.21.15.37:0/1526271536 >> 172.21.15.37:0/1526271536 conn(0x7f7be40789e0 msgr2=0x7f7be4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.532+0000 7f7beb3f7700 1 -- 172.21.15.37:0/1526271536 shutdown_connections 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 -- 172.21.15.37:0/1526271536 wait complete. 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 Processor -- start 2024-04-25T21:05:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 -- start start 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be406bbb0 0x7f7be4101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be4108a60 0x7f7be4102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4112ca0 con 0x7f7be406b1f0 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4112b20 con 0x7f7be406bbb0 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.533+0000 7f7beb3f7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7be4112e20 con 0x7f7be4108a60 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:29.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47326/0 (socket says 172.21.15.37:47326) 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 -- 172.21.15.37:0/296167520 learned_addr learned my addr 172.21.15.37:0/296167520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be8992700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be406bbb0 0x7f7be4101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9994700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be4108a60 0x7f7be4102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 -- 172.21.15.37:0/296167520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be4108a60 msgr2=0x7f7be4102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be4108a60 0x7f7be4102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 -- 172.21.15.37:0/296167520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be406bbb0 msgr2=0x7f7be4101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be406bbb0 0x7f7be4101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 -- 172.21.15.37:0/296167520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be41a7730 con 0x7f7be406b1f0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7be9193700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b530 tx=0x7f7be000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0015020 con 0x7f7be406b1f0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be41a7a20 con 0x7f7be406b1f0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.534+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be41a7f80 con 0x7f7be406b1f0 2024-04-25T21:05:29.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.535+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be0011ce0 con 0x7f7be406b1f0 2024-04-25T21:05:29.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.535+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0010e40 con 0x7f7be406b1f0 2024-04-25T21:05:29.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.535+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be410b060 con 0x7f7be406b1f0 2024-04-25T21:05:29.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.537+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7be0018020 con 0x7f7be406b1f0 2024-04-25T21:05:29.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.537+0000 7f7bda7fc700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 0x7f7bd00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:29.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.537+0000 7f7be8992700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 0x7f7bd00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:29.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.537+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f7be00cffd0 con 0x7f7be406b1f0 2024-04-25T21:05:29.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.538+0000 7f7be8992700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 0x7f7bd00a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400b5c0 tx=0x7f7bd4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:29.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.541+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7be001f020 con 0x7f7be406b1f0 2024-04-25T21:05:29.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.784+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7be4062e80 con 0x7f7bd00a3d50 2024-04-25T21:05:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.812+0000 7f7bda7fc700 1 -- 172.21.15.37:0/296167520 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7be4062e80 con 0x7f7bd00a3d50 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 msgr2=0x7f7bd00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 0x7f7bd00a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400b5c0 tx=0x7f7bd4005960 comp rx=0 tx=0).stop 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 msgr2=0x7f7be41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b530 tx=0x7f7be000b840 comp rx=0 tx=0).stop 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 shutdown_connections 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be4108a60 0x7f7be4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7be406b1f0 0x7f7be41019b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7bd00a3d50 0x7f7bd00a6210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 --2- 172.21.15.37:0/296167520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7be406bbb0 0x7f7be4101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.814+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 >> 172.21.15.37:0/296167520 conn(0x7f7be40789e0 msgr2=0x7f7be40ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.815+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 shutdown_connections 2024-04-25T21:05:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:29.815+0000 7f7beb3f7700 1 -- 172.21.15.37:0/296167520 wait complete. 2024-04-25T21:05:30.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:29 smithi177 ceph-mon[33043]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-25T21:05:30.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:29 smithi177 ceph-mon[33043]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T21:05:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:29 smithi098 ceph-mon[32569]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-25T21:05:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:29 smithi098 ceph-mon[32569]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T21:05:30.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:29 smithi037 ceph-mon[29195]: pgmap v315: 97 pgs: 97 active+clean; 584 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-25T21:05:30.218 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:29 smithi037 ceph-mon[29195]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-04-25T21:05:30.533 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T21:05:30.784 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:31.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:30 smithi037 ceph-mon[29195]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:31.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:30 smithi177 ceph-mon[33043]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:31.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:30 smithi098 ceph-mon[32569]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:31 smithi177 ceph-mon[33043]: pgmap v316: 97 pgs: 97 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 22 op/s 2024-04-25T21:05:32.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:31 smithi037 ceph-mon[29195]: pgmap v316: 97 pgs: 97 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 22 op/s 2024-04-25T21:05:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:31 smithi098 ceph-mon[32569]: pgmap v316: 97 pgs: 97 active+clean; 585 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.2 KiB/s wr, 22 op/s 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 -- 172.21.15.37:0/4205585611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8107980 msgr2=0x7faff8107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 --2- 172.21.15.37:0/4205585611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8107980 0x7faff8107d60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009990 tx=0x7fafe8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 -- 172.21.15.37:0/4205585611 shutdown_connections 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 --2- 172.21.15.37:0/4205585611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 0x7faff810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 --2- 172.21.15.37:0/4205585611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8107980 0x7faff8107d60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 --2- 172.21.15.37:0/4205585611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8101930 0x7faff8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 -- 172.21.15.37:0/4205585611 >> 172.21.15.37:0/4205585611 conn(0x7faff80788b0 msgr2=0x7faff8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 -- 172.21.15.37:0/4205585611 shutdown_connections 2024-04-25T21:05:32.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.617+0000 7faffe083700 1 -- 172.21.15.37:0/4205585611 wait complete. 2024-04-25T21:05:32.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 Processor -- start 2024-04-25T21:05:32.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 -- start start 2024-04-25T21:05:32.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:32.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 0x7faff8103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8107980 0x7faff8103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faff8111ae0 con 0x7faff8101930 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faff8111960 con 0x7faff81022f0 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faffe083700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faff8111c60 con 0x7faff8107980 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faff77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faff77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47350/0 (socket says 172.21.15.37:47350) 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faff77fe700 1 -- 172.21.15.37:0/2758165392 learned_addr learned my addr 172.21.15.37:0/2758165392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.618+0000 7faff7fff700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8107980 0x7faff8103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff6ffd700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 0x7faff8103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 -- 172.21.15.37:0/2758165392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8107980 msgr2=0x7faff8103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8107980 0x7faff8103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 -- 172.21.15.37:0/2758165392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 msgr2=0x7faff8103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 0x7faff8103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 -- 172.21.15.37:0/2758165392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff81a75b0 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff77fe700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafe8000c00 tx=0x7fafe80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8027070 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe8003f20 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff81a7840 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8022cf0 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.619+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff81a7c90 con 0x7faff8101930 2024-04-25T21:05:32.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.620+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff8069730 con 0x7faff8101930 2024-04-25T21:05:32.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.621+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fafe802a070 con 0x7faff8101930 2024-04-25T21:05:32.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.622+0000 7faff4ff9700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 0x7fafe00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:32.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.622+0000 7faff6ffd700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 0x7fafe00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:32.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.622+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fafe80d1180 con 0x7faff8101930 2024-04-25T21:05:32.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.622+0000 7faff6ffd700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 0x7fafe00a6190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafec005950 tx=0x7fafec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:32.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.626+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fafe8013070 con 0x7faff8101930 2024-04-25T21:05:32.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.870+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7faff8105620 con 0x7fafe00a3cd0 2024-04-25T21:05:32.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.875+0000 7faff4ff9700 1 -- 172.21.15.37:0/2758165392 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faff8105620 con 0x7fafe00a3cd0 2024-04-25T21:05:32.876 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/e21166e2-fcf9-4e0a-8db7-641bc408474d 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 msgr2=0x7fafe00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 0x7fafe00a6190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafec005950 tx=0x7fafec0058e0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 msgr2=0x7faff81034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fafe8000c00 tx=0x7fafe80047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 shutdown_connections 2024-04-25T21:05:32.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faff8107980 0x7faff8103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faff8101930 0x7faff81034e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fafe00a3cd0 0x7fafe00a6190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 --2- 172.21.15.37:0/2758165392 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faff81022f0 0x7faff8103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.877+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 >> 172.21.15.37:0/2758165392 conn(0x7faff80788b0 msgr2=0x7faff8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.878+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 shutdown_connections 2024-04-25T21:05:32.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:32.878+0000 7faffe083700 1 -- 172.21.15.37:0/2758165392 wait complete. 2024-04-25T21:05:33.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T21:05:33.776 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:33 smithi177 ceph-mon[33043]: pgmap v317: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-04-25T21:05:34.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:33 smithi177 ceph-mon[33043]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:34.130 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:33 smithi037 ceph-mon[29195]: pgmap v317: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-04-25T21:05:34.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:33 smithi037 ceph-mon[29195]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:33 smithi098 ceph-mon[32569]: pgmap v317: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-04-25T21:05:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:33 smithi098 ceph-mon[32569]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:34.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T21:05:34.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T21:05:34.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T21:05:34.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T21:05:34.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T21:05:34.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T21:05:34.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T21:05:34.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:34 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T21:05:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:34 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-25T21:05:35.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T21:05:35.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-04-25T21:05:35.168 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:34 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 -- 172.21.15.37:0/3314530700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49806b980 msgr2=0x7fe49810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/3314530700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49806b980 0x7fe49810f8f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b0a0 tx=0x7fe49000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 -- 172.21.15.37:0/3314530700 shutdown_connections 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/3314530700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49806b980 0x7fe49810f8f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/3314530700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe498108a20 0x7fe498108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/3314530700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806afe0 0x7fe49806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 -- 172.21.15.37:0/3314530700 >> 172.21.15.37:0/3314530700 conn(0x7fe4980788b0 msgr2=0x7fe498078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 -- 172.21.15.37:0/3314530700 shutdown_connections 2024-04-25T21:05:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.484+0000 7fe49dbc2700 1 -- 172.21.15.37:0/3314530700 wait complete. 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.485+0000 7fe49dbc2700 1 Processor -- start 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.485+0000 7fe49dbc2700 1 -- start start 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.485+0000 7fe49dbc2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 0x7fe498199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.485+0000 7fe49dbc2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 0x7fe498199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.485+0000 7fe49dbc2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe49dbc2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe498112cf0 con 0x7fe498108a20 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe49dbc2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe498112b70 con 0x7fe49806afe0 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe49dbc2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe498112e70 con 0x7fe49806b980 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47370/0 (socket says 172.21.15.37:47370) 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 -- 172.21.15.37:0/62601433 learned_addr learned my addr 172.21.15.37:0/62601433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe496ffd700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 0x7fe498199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe4977fe700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 0x7fe498199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 -- 172.21.15.37:0/62601433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 msgr2=0x7fe498199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 0x7fe498199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 -- 172.21.15.37:0/62601433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 msgr2=0x7fe498199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 0x7fe498199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 -- 172.21.15.37:0/62601433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49819e250 con 0x7fe498108a20 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe4977fe700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 0x7fe498199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe496ffd700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 0x7fe498199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe497fff700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b070 tx=0x7fe49000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490021070 con 0x7fe498108a20 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe490007cf0 con 0x7fe498108a20 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490005440 con 0x7fe498108a20 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49819e4e0 con 0x7fe498108a20 2024-04-25T21:05:35.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.486+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a5dc0 con 0x7fe498108a20 2024-04-25T21:05:35.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.487+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49810b020 con 0x7fe498108a20 2024-04-25T21:05:35.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.488+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe490024070 con 0x7fe498108a20 2024-04-25T21:05:35.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.489+0000 7fe494ff9700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 0x7fe4800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.489+0000 7fe4977fe700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 0x7fe4800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.489+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8211+0+0 (secure 0 0 0) 0x7fe49000f020 con 0x7fe498108a20 2024-04-25T21:05:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.489+0000 7fe4977fe700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 0x7fe4800a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe488007900 tx=0x7fe488008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:35.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.494+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe49002b020 con 0x7fe498108a20 2024-04-25T21:05:35.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.737+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe49819a5a0 con 0x7fe4800a3d50 2024-04-25T21:05:35.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.770+0000 7fe494ff9700 1 -- 172.21.15.37:0/62601433 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe49819a5a0 con 0x7fe4800a3d50 2024-04-25T21:05:35.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 msgr2=0x7fe4800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 0x7fe4800a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe488007900 tx=0x7fe488008040 comp rx=0 tx=0).stop 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 msgr2=0x7fe49819db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe49000b070 tx=0x7fe49000cf00 comp rx=0 tx=0).stop 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 shutdown_connections 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe498108a20 0x7fe49819db10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe4800a3d50 0x7fe4800a6210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe49806afe0 0x7fe498199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 --2- 172.21.15.37:0/62601433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe49806b980 0x7fe498199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 >> 172.21.15.37:0/62601433 conn(0x7fe4980788b0 msgr2=0x7fe4980ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 shutdown_connections 2024-04-25T21:05:35.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:35.773+0000 7fe49dbc2700 1 -- 172.21.15.37:0/62601433 wait complete. 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T21:05:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T21:05:36.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T21:05:36.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T21:05:36.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-25T21:05:36.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:35 smithi177 ceph-mon[33043]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T21:05:36.144 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T21:05:36.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-25T21:05:36.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:35 smithi037 ceph-mon[29195]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T21:05:36.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: pgmap v318: 97 pgs: 97 active+clean; 605 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-04-25T21:05:36.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T21:05:36.231 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-04-25T21:05:36.232 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:35 smithi098 ceph-mon[32569]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T21:05:36.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T21:05:36.756 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:37.081 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:36 smithi037 ceph-mon[29195]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:37.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:36 smithi037 ceph-mon[29195]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T21:05:37.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:36 smithi177 ceph-mon[33043]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:37.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:36 smithi177 ceph-mon[33043]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T21:05:37.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:36 smithi098 ceph-mon[32569]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:37.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:36 smithi098 ceph-mon[32569]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T21:05:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:37 smithi177 ceph-mon[33043]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-25T21:05:38.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:37 smithi037 ceph-mon[29195]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-25T21:05:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:37 smithi098 ceph-mon[32569]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 -- 172.21.15.37:0/3470943482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 msgr2=0x7f1254102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 --2- 172.21.15.37:0/3470943482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254102f40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1244009990 tx=0x7f1244009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 -- 172.21.15.37:0/3470943482 shutdown_connections 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 --2- 172.21.15.37:0/3470943482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254103480 0x7f125410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 --2- 172.21.15.37:0/3470943482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254108ab0 0x7f1254108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 --2- 172.21.15.37:0/3470943482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254102f40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 -- 172.21.15.37:0/3470943482 >> 172.21.15.37:0/3470943482 conn(0x7f1254079bf0 msgr2=0x7f125407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 -- 172.21.15.37:0/3470943482 shutdown_connections 2024-04-25T21:05:38.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.350+0000 7f125c50b700 1 -- 172.21.15.37:0/3470943482 wait complete. 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 Processor -- start 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 -- start start 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254103480 0x7f1254199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254108ab0 0x7f125419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1254112cb0 con 0x7f1254102ac0 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1254112b30 con 0x7f1254103480 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125c50b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1254112e30 con 0x7f1254108ab0 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125a2a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125a2a7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47386/0 (socket says 172.21.15.37:47386) 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125a2a7700 1 -- 172.21.15.37:0/1606582476 learned_addr learned my addr 172.21.15.37:0/1606582476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f1259aa6700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254103480 0x7f1254199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.351+0000 7f125aaa8700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254108ab0 0x7f125419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:38.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 -- 172.21.15.37:0/1606582476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254108ab0 msgr2=0x7f125419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254108ab0 0x7f125419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 -- 172.21.15.37:0/1606582476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254103480 msgr2=0x7f1254199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254103480 0x7f1254199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 -- 172.21.15.37:0/1606582476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125419e130 con 0x7f1254102ac0 2024-04-25T21:05:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125a2a7700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f125000d950 tx=0x7f125000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250016070 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125419e420 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f125000fd40 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125000e3b0 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.352+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541a5ca0 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.353+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1254069730 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.354+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1250019070 con 0x7f1254102ac0 2024-04-25T21:05:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.355+0000 7f124b7fe700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 0x7f12400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.355+0000 7f1259aa6700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 0x7f12400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.355+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f12500d02f0 con 0x7f1254102ac0 2024-04-25T21:05:38.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.355+0000 7f1259aa6700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 0x7f12400a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f124400b2e0 tx=0x7f1244005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.360+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f125009cb80 con 0x7f1254102ac0 2024-04-25T21:05:38.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.602+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1254062e80 con 0x7f12400a3d50 2024-04-25T21:05:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.607+0000 7f124b7fe700 1 -- 172.21.15.37:0/1606582476 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1254062e80 con 0x7f12400a3d50 2024-04-25T21:05:38.608 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/73dd5cfa-7e0d-4ed9-9126-8c1a699c1fe8 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.609+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 msgr2=0x7f12400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.609+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 0x7f12400a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f124400b2e0 tx=0x7f1244005960 comp rx=0 tx=0).stop 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.609+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 msgr2=0x7f1254199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.609+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f125000d950 tx=0x7f125000dc60 comp rx=0 tx=0).stop 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 shutdown_connections 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1254108ab0 0x7f125419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1254102ac0 0x7f1254199120 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f12400a3d50 0x7f12400a6210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 --2- 172.21.15.37:0/1606582476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1254103480 0x7f1254199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 >> 172.21.15.37:0/1606582476 conn(0x7f1254079bf0 msgr2=0x7f1254100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 shutdown_connections 2024-04-25T21:05:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:38.610+0000 7f125c50b700 1 -- 172.21.15.37:0/1606582476 wait complete. 2024-04-25T21:05:39.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:05:39.550 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:39.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:39 smithi037 ceph-mon[29195]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-04-25T21:05:39.983 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:39 smithi037 ceph-mon[29195]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:39 smithi177 ceph-mon[33043]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-04-25T21:05:40.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:39 smithi177 ceph-mon[33043]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:39 smithi098 ceph-mon[32569]: pgmap v322: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-04-25T21:05:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:39 smithi098 ceph-mon[32569]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:05:41.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/3928067487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 msgr2=0x7fe3b0108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/3928067487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b0108e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4009990 tx=0x7fe3a4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/3928067487 shutdown_connections 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/3928067487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 0x7fe3b010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/3928067487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b0108e50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/3928067487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 0x7fe3b0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/3928067487 >> 172.21.15.37:0/3928067487 conn(0x7fe3b0079bb0 msgr2=0x7fe3b0079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/3928067487 shutdown_connections 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.178+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/3928067487 wait complete. 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 Processor -- start 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 -- start start 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 0x7fe3b019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:41.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 0x7fe3b019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0112d00 con 0x7fe3b0108a70 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0112b80 con 0x7fe3b0103480 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b6b5b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0112e80 con 0x7fe3b0102ac0 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.179+0000 7fe3b50f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44176/0 (socket says 172.21.15.37:44176) 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 -- 172.21.15.37:0/206261961 learned_addr learned my addr 172.21.15.37:0/206261961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b48f7700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 0x7fe3b019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3affff700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 0x7fe3b019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 -- 172.21.15.37:0/206261961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 msgr2=0x7fe3b019b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 0x7fe3b019b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 -- 172.21.15.37:0/206261961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 msgr2=0x7fe3b019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 0x7fe3b019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 -- 172.21.15.37:0/206261961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b01a0500 con 0x7fe3b0108a70 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b50f8700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800b970 tx=0x7fe3a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8019070 con 0x7fe3b0108a70 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3a800cde0 con 0x7fe3b0108a70 2024-04-25T21:05:41.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.180+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b01a07f0 con 0x7fe3b0108a70 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.181+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8007370 con 0x7fe3b0108a70 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.181+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01a8070 con 0x7fe3b0108a70 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.182+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b01041a0 con 0x7fe3b0108a70 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.182+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe3a801c070 con 0x7fe3b0108a70 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.183+0000 7fe3adffb700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 0x7fe3a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.183+0000 7fe3b48f7700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 0x7fe3a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:41.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.183+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe3a80cf7d0 con 0x7fe3b0108a70 2024-04-25T21:05:41.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.184+0000 7fe3b48f7700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 0x7fe3a00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40098e0 tx=0x7fe3a4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:41.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.188+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe3a809c170 con 0x7fe3b0108a70 2024-04-25T21:05:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.453+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3b0069730 con 0x7fe3b0108a70 2024-04-25T21:05:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.454+0000 7fe3adffb700 1 -- 172.21.15.37:0/206261961 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7fe3a809ba70 con 0x7fe3b0108a70 2024-04-25T21:05:41.455 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:41.455 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 msgr2=0x7fe3a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 0x7fe3a00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40098e0 tx=0x7fe3a4005960 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 msgr2=0x7fe3b019fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800b970 tx=0x7fe3a800bc80 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 shutdown_connections 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0108a70 0x7fe3b019fdc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe3b0102ac0 0x7fe3b019b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe3a00a3bb0 0x7fe3a00a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 --2- 172.21.15.37:0/206261961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3b0103480 0x7fe3b019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.456+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 >> 172.21.15.37:0/206261961 conn(0x7fe3b0079bb0 msgr2=0x7fe3b010df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:41.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.457+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 shutdown_connections 2024-04-25T21:05:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:41.457+0000 7fe3b6b5b700 1 -- 172.21.15.37:0/206261961 wait complete. 2024-04-25T21:05:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:05:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:41 smithi177 ceph-mon[33043]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:42.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:41 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/206261961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:42.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:41 smithi037 ceph-mon[29195]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:42.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:41 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/206261961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:41 smithi098 ceph-mon[32569]: pgmap v323: 97 pgs: 2 peering, 95 active+clean; 624 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:41 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/206261961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:05:42.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:05:42.426 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:44.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 -- 172.21.15.37:0/565534785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec108a70 msgr2=0x7fe1ec108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:44.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/565534785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec108a70 0x7fe1ec108e50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009990 tx=0x7fe1d4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 -- 172.21.15.37:0/565534785 shutdown_connections 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/565534785 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec103480 0x7fe1ec10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/565534785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec108a70 0x7fe1ec108e50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/565534785 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec102ac0 0x7fe1ec102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 -- 172.21.15.37:0/565534785 >> 172.21.15.37:0/565534785 conn(0x7fe1ec079bb0 msgr2=0x7fe1ec079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 -- 172.21.15.37:0/565534785 shutdown_connections 2024-04-25T21:05:44.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.083+0000 7fe1f29dc700 1 -- 172.21.15.37:0/565534785 wait complete. 2024-04-25T21:05:44.085 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:43 smithi037 ceph-mon[29195]: pgmap v324: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.084+0000 7fe1f29dc700 1 Processor -- start 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.084+0000 7fe1f29dc700 1 -- start start 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec102ac0 0x7fe1ec19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 0x7fe1ec19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:44.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112c60 con 0x7fe1ec103480 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112ae0 con 0x7fe1ec108a70 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f29dc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec112de0 con 0x7fe1ec102ac0 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44190/0 (socket says 172.21.15.37:44190) 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 -- 172.21.15.37:0/1833842232 learned_addr learned my addr 172.21.15.37:0/1833842232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1ebfff700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec102ac0 0x7fe1ec19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f0f79700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 0x7fe1ec19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 -- 172.21.15.37:0/1833842232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec102ac0 msgr2=0x7fe1ec19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec102ac0 0x7fe1ec19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 -- 172.21.15.37:0/1833842232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 msgr2=0x7fe1ec19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 0x7fe1ec19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 -- 172.21.15.37:0/1833842232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec1a0270 con 0x7fe1ec103480 2024-04-25T21:05:44.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1f0f79700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 0x7fe1ec19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.085+0000 7fe1eb7fe700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00c9b0 tx=0x7fe1dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.086+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc007b00 con 0x7fe1ec103480 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.086+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1dc007ca0 con 0x7fe1ec103480 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.086+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec1a0560 con 0x7fe1ec103480 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.086+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc018660 con 0x7fe1ec103480 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.086+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1a7de0 con 0x7fe1ec103480 2024-04-25T21:05:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.087+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec069730 con 0x7fe1ec103480 2024-04-25T21:05:44.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:43 smithi177 ceph-mon[33043]: pgmap v324: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.088+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe1dc00f450 con 0x7fe1ec103480 2024-04-25T21:05:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.089+0000 7fe1e97fa700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 0x7fe1d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.089+0000 7fe1ebfff700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 0x7fe1d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.089+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe1dc0d0790 con 0x7fe1ec103480 2024-04-25T21:05:44.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.089+0000 7fe1ebfff700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 0x7fe1d80a6210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009750 tx=0x7fe1d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.092+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1dc014030 con 0x7fe1ec103480 2024-04-25T21:05:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:43 smithi098 ceph-mon[32569]: pgmap v324: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 13 op/s; 72 B/s, 0 objects/s recovering 2024-04-25T21:05:44.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.299+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1ec19c5c0 con 0x7fe1ec103480 2024-04-25T21:05:44.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.300+0000 7fe1e97fa700 1 -- 172.21.15.37:0/1833842232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22571 (secure 0 0 0) 0x7fe1ec19c5c0 con 0x7fe1ec103480 2024-04-25T21:05:44.301 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:44.301 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:35.806342+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 msgr2=0x7fe1d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 0x7fe1d80a6210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009750 tx=0x7fe1d4005fd0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 msgr2=0x7fe1ec19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00c9b0 tx=0x7fe1dc00ccc0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 shutdown_connections 2024-04-25T21:05:44.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe1d80a3d50 0x7fe1d80a6210 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1ec108a70 0x7fe1ec19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1ec102ac0 0x7fe1ec19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 --2- 172.21.15.37:0/1833842232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1ec103480 0x7fe1ec19b7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 >> 172.21.15.37:0/1833842232 conn(0x7fe1ec079bb0 msgr2=0x7fe1ec074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 shutdown_connections 2024-04-25T21:05:44.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:44.303+0000 7fe1f29dc700 1 -- 172.21.15.37:0/1833842232 wait complete. 2024-04-25T21:05:44.988 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T21:05:44.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:05:45.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:44 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1833842232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:45.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:44 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1833842232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:45.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:44 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1833842232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:05:45.240 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:45 smithi037 ceph-mon[29195]: pgmap v325: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 61 B/s, 0 objects/s recovering 2024-04-25T21:05:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:45 smithi177 ceph-mon[33043]: pgmap v325: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 61 B/s, 0 objects/s recovering 2024-04-25T21:05:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:45 smithi098 ceph-mon[32569]: pgmap v325: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 61 B/s, 0 objects/s recovering 2024-04-25T21:05:46.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.916+0000 7f8266e58700 1 -- 172.21.15.37:0/1175477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 msgr2=0x7f826010f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 --2- 172.21.15.37:0/1175477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f826010f940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f825800b0a0 tx=0x7f825800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 -- 172.21.15.37:0/1175477854 shutdown_connections 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 --2- 172.21.15.37:0/1175477854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f826010f940 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 --2- 172.21.15.37:0/1175477854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 0x7f8260108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 --2- 172.21.15.37:0/1175477854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 0x7f8260102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 -- 172.21.15.37:0/1175477854 >> 172.21.15.37:0/1175477854 conn(0x7f8260079bf0 msgr2=0x7f826007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 -- 172.21.15.37:0/1175477854 shutdown_connections 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 -- 172.21.15.37:0/1175477854 wait complete. 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 Processor -- start 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.917+0000 7f8266e58700 1 -- start start 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 0x7f8260074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:46.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 0x7f82600757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112c90 con 0x7f8260103460 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112b10 con 0x7f8260108a70 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44212/0 (socket says 172.21.15.37:44212) 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 -- 172.21.15.37:0/2873290733 learned_addr learned my addr 172.21.15.37:0/2873290733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:46.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8260112e10 con 0x7f8260102ac0 2024-04-25T21:05:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f8264bf4700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 0x7f8260074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f82653f5700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 0x7f82600757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 -- 172.21.15.37:0/2873290733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 msgr2=0x7f8260074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 0x7f8260074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 -- 172.21.15.37:0/2873290733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 msgr2=0x7f82600757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 0x7f82600757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82601a76b0 con 0x7f8260103460 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825ffff700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d980 tx=0x7f824c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.918+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c017070 con 0x7f8260103460 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.919+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f824c005020 con 0x7f8260103460 2024-04-25T21:05:46.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.919+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82601a79a0 con 0x7f8260103460 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.919+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c00fa10 con 0x7f8260103460 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.919+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82601a7ee0 con 0x7f8260103460 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.920+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8260069730 con 0x7f8260103460 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.921+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f824c004a30 con 0x7f8260103460 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.921+0000 7f825dffb700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:46.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.921+0000 7f8264bf4700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:46.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.922+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f824c0d0390 con 0x7f8260103460 2024-04-25T21:05:46.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.922+0000 7f8264bf4700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 0x7f82500a6210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8254009a10 tx=0x7f8254009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:46.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:46.926+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f824c020020 con 0x7f8260103460 2024-04-25T21:05:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.127+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8260062e80 con 0x7f82500a3d50 2024-04-25T21:05:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.135+0000 7f825dffb700 1 -- 172.21.15.37:0/2873290733 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f8260062e80 con 0x7f82500a3d50 2024-04-25T21:05:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:47.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 msgr2=0x7f82500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 0x7f82500a6210 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8254009a10 tx=0x7f8254009190 comp rx=0 tx=0).stop 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 msgr2=0x7f8260075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d980 tx=0x7f824c00dc90 comp rx=0 tx=0).stop 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 shutdown_connections 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f82500a3d50 0x7f82500a6210 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8260108a70 0x7f82600757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:47.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8260102ac0 0x7f8260074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 --2- 172.21.15.37:0/2873290733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8260103460 0x7f8260075280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 >> 172.21.15.37:0/2873290733 conn(0x7f8260079bf0 msgr2=0x7f8260100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 shutdown_connections 2024-04-25T21:05:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:47.139+0000 7f8266e58700 1 -- 172.21.15.37:0/2873290733 wait complete. 2024-04-25T21:05:47.143 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:05:47.785 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-25T21:05:46.206092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":418,"num_read_kb":331,"num_write":422,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361320,"kb_used_data":12132,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580504,"statfs":{"total":1151940427776,"available":1151570436096,"internally_reserved":0,"allocated":12423168,"data_stored":7708107,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32154,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":97,"num_write":19,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.397931"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812561+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:05:35.812561+0000","last_peered":"2024-04-25T21:05:35.812561+0000","last_clean":"2024-04-25T21:05:35.812561+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:05:35.812561+0000","last_undegraded":"2024-04-25T21:05:35.812561+0000","last_fullsized":"2024-04-25T21:05:35.812561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812177+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:05:35.812177+0000","last_peered":"2024-04-25T21:05:35.812177+0000","last_clean":"2024-04-25T21:05:35.812177+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:05:35.812177+0000","last_undegraded":"2024-04-25T21:05:35.812177+0000","last_fullsized":"2024-04-25T21:05:35.812177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811939+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:05:35.811939+0000","last_peered":"2024-04-25T21:05:35.811939+0000","last_clean":"2024-04-25T21:05:35.811939+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:05:35.811939+0000","last_undegraded":"2024-04-25T21:05:35.811939+0000","last_fullsized":"2024-04-25T21:05:35.811939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280134+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:05:36.280134+0000","last_peered":"2024-04-25T21:05:36.280134+0000","last_clean":"2024-04-25T21:05:36.280134+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:05:36.280134+0000","last_undegraded":"2024-04-25T21:05:36.280134+0000","last_fullsized":"2024-04-25T21:05:36.280134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815068+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:05:35.815068+0000","last_peered":"2024-04-25T21:05:35.815068+0000","last_clean":"2024-04-25T21:05:35.815068+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:05:35.815068+0000","last_undegraded":"2024-04-25T21:05:35.815068+0000","last_fullsized":"2024-04-25T21:05:35.815068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054645+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:05:36.054645+0000","last_peered":"2024-04-25T21:05:36.054645+0000","last_clean":"2024-04-25T21:05:36.054645+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:05:36.054645+0000","last_undegraded":"2024-04-25T21:05:36.054645+0000","last_fullsized":"2024-04-25T21:05:36.054645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810131+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:05:35.810131+0000","last_peered":"2024-04-25T21:05:35.810131+0000","last_clean":"2024-04-25T21:05:35.810131+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:05:35.810131+0000","last_undegraded":"2024-04-25T21:05:35.810131+0000","last_fullsized":"2024-04-25T21:05:35.810131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164368+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:05:36.164368+0000","last_peered":"2024-04-25T21:05:36.164368+0000","last_clean":"2024-04-25T21:05:36.164368+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:05:36.164368+0000","last_undegraded":"2024-04-25T21:05:36.164368+0000","last_fullsized":"2024-04-25T21:05:36.164368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812458+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:05:35.812458+0000","last_peered":"2024-04-25T21:05:35.812458+0000","last_clean":"2024-04-25T21:05:35.812458+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:05:35.812458+0000","last_undegraded":"2024-04-25T21:05:35.812458+0000","last_fullsized":"2024-04-25T21:05:35.812458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054978+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:05:36.054978+0000","last_peered":"2024-04-25T21:05:36.054978+0000","last_clean":"2024-04-25T21:05:36.054978+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:05:36.054978+0000","last_undegraded":"2024-04-25T21:05:36.054978+0000","last_fullsized":"2024-04-25T21:05:36.054978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810362+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:05:35.810362+0000","last_peered":"2024-04-25T21:05:35.810362+0000","last_clean":"2024-04-25T21:05:35.810362+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:05:35.810362+0000","last_undegraded":"2024-04-25T21:05:35.810362+0000","last_fullsized":"2024-04-25T21:05:35.810362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809516+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:05:35.809516+0000","last_peered":"2024-04-25T21:05:35.809516+0000","last_clean":"2024-04-25T21:05:35.809516+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:05:35.809516+0000","last_undegraded":"2024-04-25T21:05:35.809516+0000","last_fullsized":"2024-04-25T21:05:35.809516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811795+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:05:35.811795+0000","last_peered":"2024-04-25T21:05:35.811795+0000","last_clean":"2024-04-25T21:05:35.811795+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:05:35.811795+0000","last_undegraded":"2024-04-25T21:05:35.811795+0000","last_fullsized":"2024-04-25T21:05:35.811795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818318+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:05:35.818318+0000","last_peered":"2024-04-25T21:05:35.818318+0000","last_clean":"2024-04-25T21:05:35.818318+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:05:35.818318+0000","last_undegraded":"2024-04-25T21:05:35.818318+0000","last_fullsized":"2024-04-25T21:05:35.818318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811748+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:05:35.811748+0000","last_peered":"2024-04-25T21:05:35.811748+0000","last_clean":"2024-04-25T21:05:35.811748+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:05:35.811748+0000","last_undegraded":"2024-04-25T21:05:35.811748+0000","last_fullsized":"2024-04-25T21:05:35.811748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812733+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:05:35.812733+0000","last_peered":"2024-04-25T21:05:35.812733+0000","last_clean":"2024-04-25T21:05:35.812733+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:05:35.812733+0000","last_undegraded":"2024-04-25T21:05:35.812733+0000","last_fullsized":"2024-04-25T21:05:35.812733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847351+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:05:35.847351+0000","last_peered":"2024-04-25T21:05:35.847351+0000","last_clean":"2024-04-25T21:05:35.847351+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:05:35.847351+0000","last_undegraded":"2024-04-25T21:05:35.847351+0000","last_fullsized":"2024-04-25T21:05:35.847351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813305+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:05:35.813305+0000","last_peered":"2024-04-25T21:05:35.813305+0000","last_clean":"2024-04-25T21:05:35.813305+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:05:35.813305+0000","last_undegraded":"2024-04-25T21:05:35.813305+0000","last_fullsized":"2024-04-25T21:05:35.813305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:42.286332+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:05:42.286332+0000","last_peered":"2024-04-25T21:05:42.286332+0000","last_clean":"2024-04-25T21:05:42.286332+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:05:42.286332+0000","last_undegraded":"2024-04-25T21:05:42.286332+0000","last_fullsized":"2024-04-25T21:05:42.286332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164650+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:05:36.164650+0000","last_peered":"2024-04-25T21:05:36.164650+0000","last_clean":"2024-04-25T21:05:36.164650+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:05:36.164650+0000","last_undegraded":"2024-04-25T21:05:36.164650+0000","last_fullsized":"2024-04-25T21:05:36.164650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164788+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:05:36.164788+0000","last_peered":"2024-04-25T21:05:36.164788+0000","last_clean":"2024-04-25T21:05:36.164788+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:05:36.164788+0000","last_undegraded":"2024-04-25T21:05:36.164788+0000","last_fullsized":"2024-04-25T21:05:36.164788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812294+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:05:35.812294+0000","last_peered":"2024-04-25T21:05:35.812294+0000","last_clean":"2024-04-25T21:05:35.812294+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:05:35.812294+0000","last_undegraded":"2024-04-25T21:05:35.812294+0000","last_fullsized":"2024-04-25T21:05:35.812294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815926+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:05:35.815926+0000","last_peered":"2024-04-25T21:05:35.815926+0000","last_clean":"2024-04-25T21:05:35.815926+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:05:35.815926+0000","last_undegraded":"2024-04-25T21:05:35.815926+0000","last_fullsized":"2024-04-25T21:05:35.815926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811808+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:05:35.811808+0000","last_peered":"2024-04-25T21:05:35.811808+0000","last_clean":"2024-04-25T21:05:35.811808+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:05:35.811808+0000","last_undegraded":"2024-04-25T21:05:35.811808+0000","last_fullsized":"2024-04-25T21:05:35.811808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.823684+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:05:35.823684+0000","last_peered":"2024-04-25T21:05:35.823684+0000","last_clean":"2024-04-25T21:05:35.823684+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:05:35.823684+0000","last_undegraded":"2024-04-25T21:05:35.823684+0000","last_fullsized":"2024-04-25T21:05:35.823684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810659+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:05:35.810659+0000","last_peered":"2024-04-25T21:05:35.810659+0000","last_clean":"2024-04-25T21:05:35.810659+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:05:35.810659+0000","last_undegraded":"2024-04-25T21:05:35.810659+0000","last_fullsized":"2024-04-25T21:05:35.810659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:43.289138+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:05:43.289138+0000","last_peered":"2024-04-25T21:05:43.289138+0000","last_clean":"2024-04-25T21:05:43.289138+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:05:43.289138+0000","last_undegraded":"2024-04-25T21:05:43.289138+0000","last_fullsized":"2024-04-25T21:05:43.289138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815987+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:05:35.815987+0000","last_peered":"2024-04-25T21:05:35.815987+0000","last_clean":"2024-04-25T21:05:35.815987+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:05:35.815987+0000","last_undegraded":"2024-04-25T21:05:35.815987+0000","last_fullsized":"2024-04-25T21:05:35.815987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054563+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:05:36.054563+0000","last_peered":"2024-04-25T21:05:36.054563+0000","last_clean":"2024-04-25T21:05:36.054563+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:05:36.054563+0000","last_undegraded":"2024-04-25T21:05:36.054563+0000","last_fullsized":"2024-04-25T21:05:36.054563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810324+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:05:35.810324+0000","last_peered":"2024-04-25T21:05:35.810324+0000","last_clean":"2024-04-25T21:05:35.810324+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:05:35.810324+0000","last_undegraded":"2024-04-25T21:05:35.810324+0000","last_fullsized":"2024-04-25T21:05:35.810324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809323+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:05:35.809323+0000","last_peered":"2024-04-25T21:05:35.809323+0000","last_clean":"2024-04-25T21:05:35.809323+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:05:35.809323+0000","last_undegraded":"2024-04-25T21:05:35.809323+0000","last_fullsized":"2024-04-25T21:05:35.809323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164264+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:05:36.164264+0000","last_peered":"2024-04-25T21:05:36.164264+0000","last_clean":"2024-04-25T21:05:36.164264+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:05:36.164264+0000","last_undegraded":"2024-04-25T21:05:36.164264+0000","last_fullsized":"2024-04-25T21:05:36.164264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809991+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:05:35.809991+0000","last_peered":"2024-04-25T21:05:35.809991+0000","last_clean":"2024-04-25T21:05:35.809991+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:05:35.809991+0000","last_undegraded":"2024-04-25T21:05:35.809991+0000","last_fullsized":"2024-04-25T21:05:35.809991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847028+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:05:35.847028+0000","last_peered":"2024-04-25T21:05:35.847028+0000","last_clean":"2024-04-25T21:05:35.847028+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:05:35.847028+0000","last_undegraded":"2024-04-25T21:05:35.847028+0000","last_fullsized":"2024-04-25T21:05:35.847028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:45.291237+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:05:45.291237+0000","last_peered":"2024-04-25T21:05:45.291237+0000","last_clean":"2024-04-25T21:05:45.291237+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:05:45.291237+0000","last_undegraded":"2024-04-25T21:05:45.291237+0000","last_fullsized":"2024-04-25T21:05:45.291237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812371+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:05:35.812371+0000","last_peered":"2024-04-25T21:05:35.812371+0000","last_clean":"2024-04-25T21:05:35.812371+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:05:35.812371+0000","last_undegraded":"2024-04-25T21:05:35.812371+0000","last_fullsized":"2024-04-25T21:05:35.812371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810524+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:05:35.810524+0000","last_peered":"2024-04-25T21:05:35.810524+0000","last_clean":"2024-04-25T21:05:35.810524+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:05:35.810524+0000","last_undegraded":"2024-04-25T21:05:35.810524+0000","last_fullsized":"2024-04-25T21:05:35.810524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812286+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:05:35.812286+0000","last_peered":"2024-04-25T21:05:35.812286+0000","last_clean":"2024-04-25T21:05:35.812286+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:05:35.812286+0000","last_undegraded":"2024-04-25T21:05:35.812286+0000","last_fullsized":"2024-04-25T21:05:35.812286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810988+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:05:35.810988+0000","last_peered":"2024-04-25T21:05:35.810988+0000","last_clean":"2024-04-25T21:05:35.810988+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:05:35.810988+0000","last_undegraded":"2024-04-25T21:05:35.810988+0000","last_fullsized":"2024-04-25T21:05:35.810988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812018+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:05:35.812018+0000","last_peered":"2024-04-25T21:05:35.812018+0000","last_clean":"2024-04-25T21:05:35.812018+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:05:35.812018+0000","last_undegraded":"2024-04-25T21:05:35.812018+0000","last_fullsized":"2024-04-25T21:05:35.812018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'28","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:45.294397+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:05:45.294397+0000","last_peered":"2024-04-25T21:05:45.294397+0000","last_clean":"2024-04-25T21:05:45.294397+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:05:45.294397+0000","last_undegraded":"2024-04-25T21:05:45.294397+0000","last_fullsized":"2024-04-25T21:05:45.294397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":82,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815665+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:05:35.815665+0000","last_peered":"2024-04-25T21:05:35.815665+0000","last_clean":"2024-04-25T21:05:35.815665+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:05:35.815665+0000","last_undegraded":"2024-04-25T21:05:35.815665+0000","last_fullsized":"2024-04-25T21:05:35.815665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813128+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:05:35.813128+0000","last_peered":"2024-04-25T21:05:35.813128+0000","last_clean":"2024-04-25T21:05:35.813128+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:05:35.813128+0000","last_undegraded":"2024-04-25T21:05:35.813128+0000","last_fullsized":"2024-04-25T21:05:35.813128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809221+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:05:35.809221+0000","last_peered":"2024-04-25T21:05:35.809221+0000","last_clean":"2024-04-25T21:05:35.809221+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:05:35.809221+0000","last_undegraded":"2024-04-25T21:05:35.809221+0000","last_fullsized":"2024-04-25T21:05:35.809221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810239+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:05:35.810239+0000","last_peered":"2024-04-25T21:05:35.810239+0000","last_clean":"2024-04-25T21:05:35.810239+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:05:35.810239+0000","last_undegraded":"2024-04-25T21:05:35.810239+0000","last_fullsized":"2024-04-25T21:05:35.810239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848492+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:05:35.848492+0000","last_peered":"2024-04-25T21:05:35.848492+0000","last_clean":"2024-04-25T21:05:35.848492+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:05:35.848492+0000","last_undegraded":"2024-04-25T21:05:35.848492+0000","last_fullsized":"2024-04-25T21:05:35.848492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:39.284018+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:05:39.284018+0000","last_peered":"2024-04-25T21:05:39.284018+0000","last_clean":"2024-04-25T21:05:39.284018+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:05:39.284018+0000","last_undegraded":"2024-04-25T21:05:39.284018+0000","last_fullsized":"2024-04-25T21:05:39.284018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809971+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:05:35.809971+0000","last_peered":"2024-04-25T21:05:35.809971+0000","last_clean":"2024-04-25T21:05:35.809971+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:05:35.809971+0000","last_undegraded":"2024-04-25T21:05:35.809971+0000","last_fullsized":"2024-04-25T21:05:35.809971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815899+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:05:35.815899+0000","last_peered":"2024-04-25T21:05:35.815899+0000","last_clean":"2024-04-25T21:05:35.815899+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:05:35.815899+0000","last_undegraded":"2024-04-25T21:05:35.815899+0000","last_fullsized":"2024-04-25T21:05:35.815899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810151+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:05:35.810151+0000","last_peered":"2024-04-25T21:05:35.810151+0000","last_clean":"2024-04-25T21:05:35.810151+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:05:35.810151+0000","last_undegraded":"2024-04-25T21:05:35.810151+0000","last_fullsized":"2024-04-25T21:05:35.810151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815284+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:05:35.815284+0000","last_peered":"2024-04-25T21:05:35.815284+0000","last_clean":"2024-04-25T21:05:35.815284+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:05:35.815284+0000","last_undegraded":"2024-04-25T21:05:35.815284+0000","last_fullsized":"2024-04-25T21:05:35.815284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812340+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:05:35.812340+0000","last_peered":"2024-04-25T21:05:35.812340+0000","last_clean":"2024-04-25T21:05:35.812340+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:05:35.812340+0000","last_undegraded":"2024-04-25T21:05:35.812340+0000","last_fullsized":"2024-04-25T21:05:35.812340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847208+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:05:35.847208+0000","last_peered":"2024-04-25T21:05:35.847208+0000","last_clean":"2024-04-25T21:05:35.847208+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:05:35.847208+0000","last_undegraded":"2024-04-25T21:05:35.847208+0000","last_fullsized":"2024-04-25T21:05:35.847208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054796+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:05:36.054796+0000","last_peered":"2024-04-25T21:05:36.054796+0000","last_clean":"2024-04-25T21:05:36.054796+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:05:36.054796+0000","last_undegraded":"2024-04-25T21:05:36.054796+0000","last_fullsized":"2024-04-25T21:05:36.054796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:40.284429+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:05:40.284429+0000","last_peered":"2024-04-25T21:05:40.284429+0000","last_clean":"2024-04-25T21:05:40.284429+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:05:40.284429+0000","last_undegraded":"2024-04-25T21:05:40.284429+0000","last_fullsized":"2024-04-25T21:05:40.284429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054640+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:05:36.054640+0000","last_peered":"2024-04-25T21:05:36.054640+0000","last_clean":"2024-04-25T21:05:36.054640+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:05:36.054640+0000","last_undegraded":"2024-04-25T21:05:36.054640+0000","last_fullsized":"2024-04-25T21:05:36.054640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810247+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:05:35.810247+0000","last_peered":"2024-04-25T21:05:35.810247+0000","last_clean":"2024-04-25T21:05:35.810247+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:05:35.810247+0000","last_undegraded":"2024-04-25T21:05:35.810247+0000","last_fullsized":"2024-04-25T21:05:35.810247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:41.285694+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:05:41.285694+0000","last_peered":"2024-04-25T21:05:41.285694+0000","last_clean":"2024-04-25T21:05:41.285694+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:05:41.285694+0000","last_undegraded":"2024-04-25T21:05:41.285694+0000","last_fullsized":"2024-04-25T21:05:41.285694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810103+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:05:35.810103+0000","last_peered":"2024-04-25T21:05:35.810103+0000","last_clean":"2024-04-25T21:05:35.810103+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:05:35.810103+0000","last_undegraded":"2024-04-25T21:05:35.810103+0000","last_fullsized":"2024-04-25T21:05:35.810103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809159+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:05:35.809159+0000","last_peered":"2024-04-25T21:05:35.809159+0000","last_clean":"2024-04-25T21:05:35.809159+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:05:35.809159+0000","last_undegraded":"2024-04-25T21:05:35.809159+0000","last_fullsized":"2024-04-25T21:05:35.809159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810226+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:05:35.810226+0000","last_peered":"2024-04-25T21:05:35.810226+0000","last_clean":"2024-04-25T21:05:35.810226+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:05:35.810226+0000","last_undegraded":"2024-04-25T21:05:35.810226+0000","last_fullsized":"2024-04-25T21:05:35.810226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815740+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:05:35.815740+0000","last_peered":"2024-04-25T21:05:35.815740+0000","last_clean":"2024-04-25T21:05:35.815740+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:05:35.815740+0000","last_undegraded":"2024-04-25T21:05:35.815740+0000","last_fullsized":"2024-04-25T21:05:35.815740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809414+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:05:35.809414+0000","last_peered":"2024-04-25T21:05:35.809414+0000","last_clean":"2024-04-25T21:05:35.809414+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:05:35.809414+0000","last_undegraded":"2024-04-25T21:05:35.809414+0000","last_fullsized":"2024-04-25T21:05:35.809414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:39.674360+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:05:39.674360+0000","last_peered":"2024-04-25T21:05:39.674360+0000","last_clean":"2024-04-25T21:05:39.674360+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:05:39.674360+0000","last_undegraded":"2024-04-25T21:05:39.674360+0000","last_fullsized":"2024-04-25T21:05:39.674360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164820+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:05:36.164820+0000","last_peered":"2024-04-25T21:05:36.164820+0000","last_clean":"2024-04-25T21:05:36.164820+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:05:36.164820+0000","last_undegraded":"2024-04-25T21:05:36.164820+0000","last_fullsized":"2024-04-25T21:05:36.164820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280051+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:05:36.280051+0000","last_peered":"2024-04-25T21:05:36.280051+0000","last_clean":"2024-04-25T21:05:36.280051+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:05:36.280051+0000","last_undegraded":"2024-04-25T21:05:36.280051+0000","last_fullsized":"2024-04-25T21:05:36.280051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812196+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:05:35.812196+0000","last_peered":"2024-04-25T21:05:35.812196+0000","last_clean":"2024-04-25T21:05:35.812196+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:05:35.812196+0000","last_undegraded":"2024-04-25T21:05:35.812196+0000","last_fullsized":"2024-04-25T21:05:35.812196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.820411+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:05:35.820411+0000","last_peered":"2024-04-25T21:05:35.820411+0000","last_clean":"2024-04-25T21:05:35.820411+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:05:35.820411+0000","last_undegraded":"2024-04-25T21:05:35.820411+0000","last_fullsized":"2024-04-25T21:05:35.820411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.816311+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:05:35.816311+0000","last_peered":"2024-04-25T21:05:35.816311+0000","last_clean":"2024-04-25T21:05:35.816311+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:05:35.816311+0000","last_undegraded":"2024-04-25T21:05:35.816311+0000","last_fullsized":"2024-04-25T21:05:35.816311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164359+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:05:36.164359+0000","last_peered":"2024-04-25T21:05:36.164359+0000","last_clean":"2024-04-25T21:05:36.164359+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:05:36.164359+0000","last_undegraded":"2024-04-25T21:05:36.164359+0000","last_fullsized":"2024-04-25T21:05:36.164359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813860+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:05:35.813860+0000","last_peered":"2024-04-25T21:05:35.813860+0000","last_clean":"2024-04-25T21:05:35.813860+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:05:35.813860+0000","last_undegraded":"2024-04-25T21:05:35.813860+0000","last_fullsized":"2024-04-25T21:05:35.813860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812840+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:05:35.812840+0000","last_peered":"2024-04-25T21:05:35.812840+0000","last_clean":"2024-04-25T21:05:35.812840+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:05:35.812840+0000","last_undegraded":"2024-04-25T21:05:35.812840+0000","last_fullsized":"2024-04-25T21:05:35.812840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815193+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:05:35.815193+0000","last_peered":"2024-04-25T21:05:35.815193+0000","last_clean":"2024-04-25T21:05:35.815193+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:05:35.815193+0000","last_undegraded":"2024-04-25T21:05:35.815193+0000","last_fullsized":"2024-04-25T21:05:35.815193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810529+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:05:35.810529+0000","last_peered":"2024-04-25T21:05:35.810529+0000","last_clean":"2024-04-25T21:05:35.810529+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:05:35.810529+0000","last_undegraded":"2024-04-25T21:05:35.810529+0000","last_fullsized":"2024-04-25T21:05:35.810529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:43.291730+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:05:43.291730+0000","last_peered":"2024-04-25T21:05:43.291730+0000","last_clean":"2024-04-25T21:05:43.291730+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:05:43.291730+0000","last_undegraded":"2024-04-25T21:05:43.291730+0000","last_fullsized":"2024-04-25T21:05:43.291730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810542+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:05:35.810542+0000","last_peered":"2024-04-25T21:05:35.810542+0000","last_clean":"2024-04-25T21:05:35.810542+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:05:35.810542+0000","last_undegraded":"2024-04-25T21:05:35.810542+0000","last_fullsized":"2024-04-25T21:05:35.810542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810498+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:05:35.810498+0000","last_peered":"2024-04-25T21:05:35.810498+0000","last_clean":"2024-04-25T21:05:35.810498+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:05:35.810498+0000","last_undegraded":"2024-04-25T21:05:35.810498+0000","last_fullsized":"2024-04-25T21:05:35.810498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810354+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:05:35.810354+0000","last_peered":"2024-04-25T21:05:35.810354+0000","last_clean":"2024-04-25T21:05:35.810354+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:05:35.810354+0000","last_undegraded":"2024-04-25T21:05:35.810354+0000","last_fullsized":"2024-04-25T21:05:35.810354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809534+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:05:35.809534+0000","last_peered":"2024-04-25T21:05:35.809534+0000","last_clean":"2024-04-25T21:05:35.809534+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:05:35.809534+0000","last_undegraded":"2024-04-25T21:05:35.809534+0000","last_fullsized":"2024-04-25T21:05:35.809534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810657+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:05:35.810657+0000","last_peered":"2024-04-25T21:05:35.810657+0000","last_clean":"2024-04-25T21:05:35.810657+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:05:35.810657+0000","last_undegraded":"2024-04-25T21:05:35.810657+0000","last_fullsized":"2024-04-25T21:05:35.810657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848543+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:05:35.848543+0000","last_peered":"2024-04-25T21:05:35.848543+0000","last_clean":"2024-04-25T21:05:35.848543+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:05:35.848543+0000","last_undegraded":"2024-04-25T21:05:35.848543+0000","last_fullsized":"2024-04-25T21:05:35.848543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.814755+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:05:35.814755+0000","last_peered":"2024-04-25T21:05:35.814755+0000","last_clean":"2024-04-25T21:05:35.814755+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:05:35.814755+0000","last_undegraded":"2024-04-25T21:05:35.814755+0000","last_fullsized":"2024-04-25T21:05:35.814755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812713+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:05:35.812713+0000","last_peered":"2024-04-25T21:05:35.812713+0000","last_clean":"2024-04-25T21:05:35.812713+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:05:35.812713+0000","last_undegraded":"2024-04-25T21:05:35.812713+0000","last_fullsized":"2024-04-25T21:05:35.812713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811944+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:05:35.811944+0000","last_peered":"2024-04-25T21:05:35.811944+0000","last_clean":"2024-04-25T21:05:35.811944+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:05:35.811944+0000","last_undegraded":"2024-04-25T21:05:35.811944+0000","last_fullsized":"2024-04-25T21:05:35.811944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809423+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:05:35.809423+0000","last_peered":"2024-04-25T21:05:35.809423+0000","last_clean":"2024-04-25T21:05:35.809423+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:05:35.809423+0000","last_undegraded":"2024-04-25T21:05:35.809423+0000","last_fullsized":"2024-04-25T21:05:35.809423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848642+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:05:35.848642+0000","last_peered":"2024-04-25T21:05:35.848642+0000","last_clean":"2024-04-25T21:05:35.848642+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:05:35.848642+0000","last_undegraded":"2024-04-25T21:05:35.848642+0000","last_fullsized":"2024-04-25T21:05:35.848642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812987+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:05:35.812987+0000","last_peered":"2024-04-25T21:05:35.812987+0000","last_clean":"2024-04-25T21:05:35.812987+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:05:35.812987+0000","last_undegraded":"2024-04-25T21:05:35.812987+0000","last_fullsized":"2024-04-25T21:05:35.812987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164713+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:05:36.164713+0000","last_peered":"2024-04-25T21:05:36.164713+0000","last_clean":"2024-04-25T21:05:36.164713+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:05:36.164713+0000","last_undegraded":"2024-04-25T21:05:36.164713+0000","last_fullsized":"2024-04-25T21:05:36.164713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810024+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:05:35.810024+0000","last_peered":"2024-04-25T21:05:35.810024+0000","last_clean":"2024-04-25T21:05:35.810024+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:05:35.810024+0000","last_undegraded":"2024-04-25T21:05:35.810024+0000","last_fullsized":"2024-04-25T21:05:35.810024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164285+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:05:36.164285+0000","last_peered":"2024-04-25T21:05:36.164285+0000","last_clean":"2024-04-25T21:05:36.164285+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:05:36.164285+0000","last_undegraded":"2024-04-25T21:05:36.164285+0000","last_fullsized":"2024-04-25T21:05:36.164285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818506+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:05:35.818506+0000","last_peered":"2024-04-25T21:05:35.818506+0000","last_clean":"2024-04-25T21:05:35.818506+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:05:35.818506+0000","last_undegraded":"2024-04-25T21:05:35.818506+0000","last_fullsized":"2024-04-25T21:05:35.818506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809406+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:05:35.809406+0000","last_peered":"2024-04-25T21:05:35.809406+0000","last_clean":"2024-04-25T21:05:35.809406+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:05:35.809406+0000","last_undegraded":"2024-04-25T21:05:35.809406+0000","last_fullsized":"2024-04-25T21:05:35.809406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813376+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:05:35.813376+0000","last_peered":"2024-04-25T21:05:35.813376+0000","last_clean":"2024-04-25T21:05:35.813376+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:05:35.813376+0000","last_undegraded":"2024-04-25T21:05:35.813376+0000","last_fullsized":"2024-04-25T21:05:35.813376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246244,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":282,"num_read_kb":212,"num_write":239,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":184713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":481464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1125887,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070504,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":480217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1073317,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":535418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":481868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":536420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":479999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072995,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:05:47.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:05:48.037 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:48.159 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:47 smithi037 ceph-mon[29195]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.6 KiB/s wr, 13 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T21:05:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:47 smithi098 ceph-mon[32569]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.6 KiB/s wr, 13 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T21:05:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:47 smithi177 ceph-mon[33043]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.6 KiB/s wr, 13 op/s; 55 B/s, 0 objects/s recovering 2024-04-25T21:05:49.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:48 smithi037 ceph-mon[29195]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:49.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:48 smithi098 ceph-mon[32569]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:48 smithi177 ceph-mon[33043]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 -- 172.21.15.37:0/2472389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9904108a60 msgr2=0x7f9904108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2472389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9904108a60 0x7f9904108e40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009990 tx=0x7f98f4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 -- 172.21.15.37:0/2472389071 shutdown_connections 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2472389071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 0x7f990410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2472389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9904108a60 0x7f9904108e40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2472389071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f990406b1f0 0x7f990406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 -- 172.21.15.37:0/2472389071 >> 172.21.15.37:0/2472389071 conn(0x7f99040789e0 msgr2=0x7f9904078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 -- 172.21.15.37:0/2472389071 shutdown_connections 2024-04-25T21:05:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.741+0000 7f9909ac3700 1 -- 172.21.15.37:0/2472389071 wait complete. 2024-04-25T21:05:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 Processor -- start 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 -- start start 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 0x7f9904199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 0x7f990419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9904112c60 con 0x7f990406b1f0 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9904112ae0 con 0x7f9904108a60 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.742+0000 7f9909ac3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9904112de0 con 0x7f990406bbb0 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34666/0 (socket says 172.21.15.37:34666) 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 -- 172.21.15.37:0/2208430656 learned_addr learned my addr 172.21.15.37:0/2208430656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9902ffd700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 0x7f9904199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9903fff700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 0x7f990419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 -- 172.21.15.37:0/2208430656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 msgr2=0x7f9904199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 0x7f9904199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 -- 172.21.15.37:0/2208430656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 msgr2=0x7f990419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 0x7f990419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 -- 172.21.15.37:0/2208430656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990419e0e0 con 0x7f990406b1f0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9903fff700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 0x7f990419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f99037fe700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4027070 con 0x7f990406b1f0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990419e370 con 0x7f990406b1f0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f4022470 con 0x7f990406b1f0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4017650 con 0x7f990406b1f0 2024-04-25T21:05:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.743+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a5c50 con 0x7f990406b1f0 2024-04-25T21:05:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.744+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990410b060 con 0x7f990406b1f0 2024-04-25T21:05:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.745+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f98f402a070 con 0x7f990406b1f0 2024-04-25T21:05:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.746+0000 7f9900ff9700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 0x7f98ec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.746+0000 7f9902ffd700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 0x7f98ec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.746+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f98f40d1390 con 0x7f990406b1f0 2024-04-25T21:05:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.747+0000 7f9902ffd700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 0x7f98ec0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9904078f50 tx=0x7f98f800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:49.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.751+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98f4013070 con 0x7f990406b1f0 2024-04-25T21:05:49.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.955+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f990419a430 con 0x7f98ec0a3cd0 2024-04-25T21:05:49.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.963+0000 7f9900ff9700 1 -- 172.21.15.37:0/2208430656 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f990419a430 con 0x7f98ec0a3cd0 2024-04-25T21:05:49.967 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 msgr2=0x7f98ec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 0x7f98ec0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9904078f50 tx=0x7f98f800a380 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 msgr2=0x7f99041990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f98f4000c00 tx=0x7f98f4009580 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 shutdown_connections 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f98ec0a3cd0 0x7f98ec0a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9904108a60 0x7f990419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f990406b1f0 0x7f99041990d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 --2- 172.21.15.37:0/2208430656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f990406bbb0 0x7f9904199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:49.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 >> 172.21.15.37:0/2208430656 conn(0x7f99040789e0 msgr2=0x7f99040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 shutdown_connections 2024-04-25T21:05:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:49.966+0000 7f9909ac3700 1 -- 172.21.15.37:0/2208430656 wait complete. 2024-04-25T21:05:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:05:49.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:49 smithi037 ceph-mon[29195]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:49 smithi098 ceph-mon[32569]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:50.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:49 smithi177 ceph-mon[33043]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:50.662 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-25T21:05:48.206657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":433,"num_read_kb":336,"num_write":422,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361320,"kb_used_data":12132,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580504,"statfs":{"total":1151940427776,"available":1151570436096,"internally_reserved":0,"allocated":12423168,"data_stored":7708107,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13421,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":82,"num_write":14,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003403"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812561+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:05:35.812561+0000","last_peered":"2024-04-25T21:05:35.812561+0000","last_clean":"2024-04-25T21:05:35.812561+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:05:35.812561+0000","last_undegraded":"2024-04-25T21:05:35.812561+0000","last_fullsized":"2024-04-25T21:05:35.812561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812177+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:05:35.812177+0000","last_peered":"2024-04-25T21:05:35.812177+0000","last_clean":"2024-04-25T21:05:35.812177+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:05:35.812177+0000","last_undegraded":"2024-04-25T21:05:35.812177+0000","last_fullsized":"2024-04-25T21:05:35.812177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811939+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:05:35.811939+0000","last_peered":"2024-04-25T21:05:35.811939+0000","last_clean":"2024-04-25T21:05:35.811939+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:05:35.811939+0000","last_undegraded":"2024-04-25T21:05:35.811939+0000","last_fullsized":"2024-04-25T21:05:35.811939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280134+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:05:36.280134+0000","last_peered":"2024-04-25T21:05:36.280134+0000","last_clean":"2024-04-25T21:05:36.280134+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:05:36.280134+0000","last_undegraded":"2024-04-25T21:05:36.280134+0000","last_fullsized":"2024-04-25T21:05:36.280134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815068+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:05:35.815068+0000","last_peered":"2024-04-25T21:05:35.815068+0000","last_clean":"2024-04-25T21:05:35.815068+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:05:35.815068+0000","last_undegraded":"2024-04-25T21:05:35.815068+0000","last_fullsized":"2024-04-25T21:05:35.815068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054645+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:05:36.054645+0000","last_peered":"2024-04-25T21:05:36.054645+0000","last_clean":"2024-04-25T21:05:36.054645+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:05:36.054645+0000","last_undegraded":"2024-04-25T21:05:36.054645+0000","last_fullsized":"2024-04-25T21:05:36.054645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810131+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:05:35.810131+0000","last_peered":"2024-04-25T21:05:35.810131+0000","last_clean":"2024-04-25T21:05:35.810131+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:05:35.810131+0000","last_undegraded":"2024-04-25T21:05:35.810131+0000","last_fullsized":"2024-04-25T21:05:35.810131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164368+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:05:36.164368+0000","last_peered":"2024-04-25T21:05:36.164368+0000","last_clean":"2024-04-25T21:05:36.164368+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:05:36.164368+0000","last_undegraded":"2024-04-25T21:05:36.164368+0000","last_fullsized":"2024-04-25T21:05:36.164368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812458+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:05:35.812458+0000","last_peered":"2024-04-25T21:05:35.812458+0000","last_clean":"2024-04-25T21:05:35.812458+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:05:35.812458+0000","last_undegraded":"2024-04-25T21:05:35.812458+0000","last_fullsized":"2024-04-25T21:05:35.812458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054978+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:05:36.054978+0000","last_peered":"2024-04-25T21:05:36.054978+0000","last_clean":"2024-04-25T21:05:36.054978+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:05:36.054978+0000","last_undegraded":"2024-04-25T21:05:36.054978+0000","last_fullsized":"2024-04-25T21:05:36.054978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810362+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:05:35.810362+0000","last_peered":"2024-04-25T21:05:35.810362+0000","last_clean":"2024-04-25T21:05:35.810362+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:05:35.810362+0000","last_undegraded":"2024-04-25T21:05:35.810362+0000","last_fullsized":"2024-04-25T21:05:35.810362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809516+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:05:35.809516+0000","last_peered":"2024-04-25T21:05:35.809516+0000","last_clean":"2024-04-25T21:05:35.809516+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:05:35.809516+0000","last_undegraded":"2024-04-25T21:05:35.809516+0000","last_fullsized":"2024-04-25T21:05:35.809516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811795+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:05:35.811795+0000","last_peered":"2024-04-25T21:05:35.811795+0000","last_clean":"2024-04-25T21:05:35.811795+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:05:35.811795+0000","last_undegraded":"2024-04-25T21:05:35.811795+0000","last_fullsized":"2024-04-25T21:05:35.811795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818318+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:05:35.818318+0000","last_peered":"2024-04-25T21:05:35.818318+0000","last_clean":"2024-04-25T21:05:35.818318+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:05:35.818318+0000","last_undegraded":"2024-04-25T21:05:35.818318+0000","last_fullsized":"2024-04-25T21:05:35.818318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811748+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:05:35.811748+0000","last_peered":"2024-04-25T21:05:35.811748+0000","last_clean":"2024-04-25T21:05:35.811748+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:05:35.811748+0000","last_undegraded":"2024-04-25T21:05:35.811748+0000","last_fullsized":"2024-04-25T21:05:35.811748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812733+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:05:35.812733+0000","last_peered":"2024-04-25T21:05:35.812733+0000","last_clean":"2024-04-25T21:05:35.812733+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:05:35.812733+0000","last_undegraded":"2024-04-25T21:05:35.812733+0000","last_fullsized":"2024-04-25T21:05:35.812733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847351+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:05:35.847351+0000","last_peered":"2024-04-25T21:05:35.847351+0000","last_clean":"2024-04-25T21:05:35.847351+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:05:35.847351+0000","last_undegraded":"2024-04-25T21:05:35.847351+0000","last_fullsized":"2024-04-25T21:05:35.847351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813305+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:05:35.813305+0000","last_peered":"2024-04-25T21:05:35.813305+0000","last_clean":"2024-04-25T21:05:35.813305+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:05:35.813305+0000","last_undegraded":"2024-04-25T21:05:35.813305+0000","last_fullsized":"2024-04-25T21:05:35.813305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":64,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:42.286332+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:05:42.286332+0000","last_peered":"2024-04-25T21:05:42.286332+0000","last_clean":"2024-04-25T21:05:42.286332+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:05:42.286332+0000","last_undegraded":"2024-04-25T21:05:42.286332+0000","last_fullsized":"2024-04-25T21:05:42.286332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164650+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:05:36.164650+0000","last_peered":"2024-04-25T21:05:36.164650+0000","last_clean":"2024-04-25T21:05:36.164650+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:05:36.164650+0000","last_undegraded":"2024-04-25T21:05:36.164650+0000","last_fullsized":"2024-04-25T21:05:36.164650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164788+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:05:36.164788+0000","last_peered":"2024-04-25T21:05:36.164788+0000","last_clean":"2024-04-25T21:05:36.164788+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:05:36.164788+0000","last_undegraded":"2024-04-25T21:05:36.164788+0000","last_fullsized":"2024-04-25T21:05:36.164788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812294+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:05:35.812294+0000","last_peered":"2024-04-25T21:05:35.812294+0000","last_clean":"2024-04-25T21:05:35.812294+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:05:35.812294+0000","last_undegraded":"2024-04-25T21:05:35.812294+0000","last_fullsized":"2024-04-25T21:05:35.812294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815926+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:05:35.815926+0000","last_peered":"2024-04-25T21:05:35.815926+0000","last_clean":"2024-04-25T21:05:35.815926+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:05:35.815926+0000","last_undegraded":"2024-04-25T21:05:35.815926+0000","last_fullsized":"2024-04-25T21:05:35.815926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811808+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:05:35.811808+0000","last_peered":"2024-04-25T21:05:35.811808+0000","last_clean":"2024-04-25T21:05:35.811808+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:05:35.811808+0000","last_undegraded":"2024-04-25T21:05:35.811808+0000","last_fullsized":"2024-04-25T21:05:35.811808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.823684+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:05:35.823684+0000","last_peered":"2024-04-25T21:05:35.823684+0000","last_clean":"2024-04-25T21:05:35.823684+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:05:35.823684+0000","last_undegraded":"2024-04-25T21:05:35.823684+0000","last_fullsized":"2024-04-25T21:05:35.823684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810659+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:05:35.810659+0000","last_peered":"2024-04-25T21:05:35.810659+0000","last_clean":"2024-04-25T21:05:35.810659+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:05:35.810659+0000","last_undegraded":"2024-04-25T21:05:35.810659+0000","last_fullsized":"2024-04-25T21:05:35.810659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:43.289138+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:05:43.289138+0000","last_peered":"2024-04-25T21:05:43.289138+0000","last_clean":"2024-04-25T21:05:43.289138+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:05:43.289138+0000","last_undegraded":"2024-04-25T21:05:43.289138+0000","last_fullsized":"2024-04-25T21:05:43.289138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815987+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:05:35.815987+0000","last_peered":"2024-04-25T21:05:35.815987+0000","last_clean":"2024-04-25T21:05:35.815987+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:05:35.815987+0000","last_undegraded":"2024-04-25T21:05:35.815987+0000","last_fullsized":"2024-04-25T21:05:35.815987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054563+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:05:36.054563+0000","last_peered":"2024-04-25T21:05:36.054563+0000","last_clean":"2024-04-25T21:05:36.054563+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:05:36.054563+0000","last_undegraded":"2024-04-25T21:05:36.054563+0000","last_fullsized":"2024-04-25T21:05:36.054563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810324+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:05:35.810324+0000","last_peered":"2024-04-25T21:05:35.810324+0000","last_clean":"2024-04-25T21:05:35.810324+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:05:35.810324+0000","last_undegraded":"2024-04-25T21:05:35.810324+0000","last_fullsized":"2024-04-25T21:05:35.810324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809323+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:05:35.809323+0000","last_peered":"2024-04-25T21:05:35.809323+0000","last_clean":"2024-04-25T21:05:35.809323+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:05:35.809323+0000","last_undegraded":"2024-04-25T21:05:35.809323+0000","last_fullsized":"2024-04-25T21:05:35.809323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164264+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:05:36.164264+0000","last_peered":"2024-04-25T21:05:36.164264+0000","last_clean":"2024-04-25T21:05:36.164264+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:05:36.164264+0000","last_undegraded":"2024-04-25T21:05:36.164264+0000","last_fullsized":"2024-04-25T21:05:36.164264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809991+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:05:35.809991+0000","last_peered":"2024-04-25T21:05:35.809991+0000","last_clean":"2024-04-25T21:05:35.809991+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:05:35.809991+0000","last_undegraded":"2024-04-25T21:05:35.809991+0000","last_fullsized":"2024-04-25T21:05:35.809991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847028+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:05:35.847028+0000","last_peered":"2024-04-25T21:05:35.847028+0000","last_clean":"2024-04-25T21:05:35.847028+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:05:35.847028+0000","last_undegraded":"2024-04-25T21:05:35.847028+0000","last_fullsized":"2024-04-25T21:05:35.847028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":82,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:45.291237+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:05:45.291237+0000","last_peered":"2024-04-25T21:05:45.291237+0000","last_clean":"2024-04-25T21:05:45.291237+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:05:45.291237+0000","last_undegraded":"2024-04-25T21:05:45.291237+0000","last_fullsized":"2024-04-25T21:05:45.291237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812371+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:05:35.812371+0000","last_peered":"2024-04-25T21:05:35.812371+0000","last_clean":"2024-04-25T21:05:35.812371+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:05:35.812371+0000","last_undegraded":"2024-04-25T21:05:35.812371+0000","last_fullsized":"2024-04-25T21:05:35.812371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810524+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:05:35.810524+0000","last_peered":"2024-04-25T21:05:35.810524+0000","last_clean":"2024-04-25T21:05:35.810524+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:05:35.810524+0000","last_undegraded":"2024-04-25T21:05:35.810524+0000","last_fullsized":"2024-04-25T21:05:35.810524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812286+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:05:35.812286+0000","last_peered":"2024-04-25T21:05:35.812286+0000","last_clean":"2024-04-25T21:05:35.812286+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:05:35.812286+0000","last_undegraded":"2024-04-25T21:05:35.812286+0000","last_fullsized":"2024-04-25T21:05:35.812286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810988+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:05:35.810988+0000","last_peered":"2024-04-25T21:05:35.810988+0000","last_clean":"2024-04-25T21:05:35.810988+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:05:35.810988+0000","last_undegraded":"2024-04-25T21:05:35.810988+0000","last_fullsized":"2024-04-25T21:05:35.810988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812018+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:05:35.812018+0000","last_peered":"2024-04-25T21:05:35.812018+0000","last_clean":"2024-04-25T21:05:35.812018+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:05:35.812018+0000","last_undegraded":"2024-04-25T21:05:35.812018+0000","last_fullsized":"2024-04-25T21:05:35.812018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'28","reported_seq":113,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:45.294397+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:05:45.294397+0000","last_peered":"2024-04-25T21:05:45.294397+0000","last_clean":"2024-04-25T21:05:45.294397+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:05:45.294397+0000","last_undegraded":"2024-04-25T21:05:45.294397+0000","last_fullsized":"2024-04-25T21:05:45.294397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":59,"num_read_kb":82,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815665+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:05:35.815665+0000","last_peered":"2024-04-25T21:05:35.815665+0000","last_clean":"2024-04-25T21:05:35.815665+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:05:35.815665+0000","last_undegraded":"2024-04-25T21:05:35.815665+0000","last_fullsized":"2024-04-25T21:05:35.815665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813128+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:05:35.813128+0000","last_peered":"2024-04-25T21:05:35.813128+0000","last_clean":"2024-04-25T21:05:35.813128+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:05:35.813128+0000","last_undegraded":"2024-04-25T21:05:35.813128+0000","last_fullsized":"2024-04-25T21:05:35.813128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809221+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:05:35.809221+0000","last_peered":"2024-04-25T21:05:35.809221+0000","last_clean":"2024-04-25T21:05:35.809221+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:05:35.809221+0000","last_undegraded":"2024-04-25T21:05:35.809221+0000","last_fullsized":"2024-04-25T21:05:35.809221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810239+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:05:35.810239+0000","last_peered":"2024-04-25T21:05:35.810239+0000","last_clean":"2024-04-25T21:05:35.810239+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:05:35.810239+0000","last_undegraded":"2024-04-25T21:05:35.810239+0000","last_fullsized":"2024-04-25T21:05:35.810239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848492+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:05:35.848492+0000","last_peered":"2024-04-25T21:05:35.848492+0000","last_clean":"2024-04-25T21:05:35.848492+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:05:35.848492+0000","last_undegraded":"2024-04-25T21:05:35.848492+0000","last_fullsized":"2024-04-25T21:05:35.848492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:44.289813+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:05:44.289813+0000","last_peered":"2024-04-25T21:05:44.289813+0000","last_clean":"2024-04-25T21:05:44.289813+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:05:44.289813+0000","last_undegraded":"2024-04-25T21:05:44.289813+0000","last_fullsized":"2024-04-25T21:05:44.289813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809971+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:05:35.809971+0000","last_peered":"2024-04-25T21:05:35.809971+0000","last_clean":"2024-04-25T21:05:35.809971+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:05:35.809971+0000","last_undegraded":"2024-04-25T21:05:35.809971+0000","last_fullsized":"2024-04-25T21:05:35.809971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815899+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:05:35.815899+0000","last_peered":"2024-04-25T21:05:35.815899+0000","last_clean":"2024-04-25T21:05:35.815899+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:05:35.815899+0000","last_undegraded":"2024-04-25T21:05:35.815899+0000","last_fullsized":"2024-04-25T21:05:35.815899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810151+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:05:35.810151+0000","last_peered":"2024-04-25T21:05:35.810151+0000","last_clean":"2024-04-25T21:05:35.810151+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:05:35.810151+0000","last_undegraded":"2024-04-25T21:05:35.810151+0000","last_fullsized":"2024-04-25T21:05:35.810151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815284+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:05:35.815284+0000","last_peered":"2024-04-25T21:05:35.815284+0000","last_clean":"2024-04-25T21:05:35.815284+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:05:35.815284+0000","last_undegraded":"2024-04-25T21:05:35.815284+0000","last_fullsized":"2024-04-25T21:05:35.815284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812340+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:05:35.812340+0000","last_peered":"2024-04-25T21:05:35.812340+0000","last_clean":"2024-04-25T21:05:35.812340+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:05:35.812340+0000","last_undegraded":"2024-04-25T21:05:35.812340+0000","last_fullsized":"2024-04-25T21:05:35.812340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847208+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:05:35.847208+0000","last_peered":"2024-04-25T21:05:35.847208+0000","last_clean":"2024-04-25T21:05:35.847208+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:05:35.847208+0000","last_undegraded":"2024-04-25T21:05:35.847208+0000","last_fullsized":"2024-04-25T21:05:35.847208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054796+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:05:36.054796+0000","last_peered":"2024-04-25T21:05:36.054796+0000","last_clean":"2024-04-25T21:05:36.054796+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:05:36.054796+0000","last_undegraded":"2024-04-25T21:05:36.054796+0000","last_fullsized":"2024-04-25T21:05:36.054796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:45.290509+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:05:45.290509+0000","last_peered":"2024-04-25T21:05:45.290509+0000","last_clean":"2024-04-25T21:05:45.290509+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:05:45.290509+0000","last_undegraded":"2024-04-25T21:05:45.290509+0000","last_fullsized":"2024-04-25T21:05:45.290509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054640+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:05:36.054640+0000","last_peered":"2024-04-25T21:05:36.054640+0000","last_clean":"2024-04-25T21:05:36.054640+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:05:36.054640+0000","last_undegraded":"2024-04-25T21:05:36.054640+0000","last_fullsized":"2024-04-25T21:05:36.054640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810247+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:05:35.810247+0000","last_peered":"2024-04-25T21:05:35.810247+0000","last_clean":"2024-04-25T21:05:35.810247+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:05:35.810247+0000","last_undegraded":"2024-04-25T21:05:35.810247+0000","last_fullsized":"2024-04-25T21:05:35.810247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:46.291573+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:05:46.291573+0000","last_peered":"2024-04-25T21:05:46.291573+0000","last_clean":"2024-04-25T21:05:46.291573+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:05:46.291573+0000","last_undegraded":"2024-04-25T21:05:46.291573+0000","last_fullsized":"2024-04-25T21:05:46.291573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810103+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:05:35.810103+0000","last_peered":"2024-04-25T21:05:35.810103+0000","last_clean":"2024-04-25T21:05:35.810103+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:05:35.810103+0000","last_undegraded":"2024-04-25T21:05:35.810103+0000","last_fullsized":"2024-04-25T21:05:35.810103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809159+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:05:35.809159+0000","last_peered":"2024-04-25T21:05:35.809159+0000","last_clean":"2024-04-25T21:05:35.809159+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:05:35.809159+0000","last_undegraded":"2024-04-25T21:05:35.809159+0000","last_fullsized":"2024-04-25T21:05:35.809159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810226+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:05:35.810226+0000","last_peered":"2024-04-25T21:05:35.810226+0000","last_clean":"2024-04-25T21:05:35.810226+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:05:35.810226+0000","last_undegraded":"2024-04-25T21:05:35.810226+0000","last_fullsized":"2024-04-25T21:05:35.810226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815740+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:05:35.815740+0000","last_peered":"2024-04-25T21:05:35.815740+0000","last_clean":"2024-04-25T21:05:35.815740+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:05:35.815740+0000","last_undegraded":"2024-04-25T21:05:35.815740+0000","last_fullsized":"2024-04-25T21:05:35.815740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809414+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:05:35.809414+0000","last_peered":"2024-04-25T21:05:35.809414+0000","last_clean":"2024-04-25T21:05:35.809414+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:05:35.809414+0000","last_undegraded":"2024-04-25T21:05:35.809414+0000","last_fullsized":"2024-04-25T21:05:35.809414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'14","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:39.674360+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:05:39.674360+0000","last_peered":"2024-04-25T21:05:39.674360+0000","last_clean":"2024-04-25T21:05:39.674360+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:05:39.674360+0000","last_undegraded":"2024-04-25T21:05:39.674360+0000","last_fullsized":"2024-04-25T21:05:39.674360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164820+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:05:36.164820+0000","last_peered":"2024-04-25T21:05:36.164820+0000","last_clean":"2024-04-25T21:05:36.164820+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:05:36.164820+0000","last_undegraded":"2024-04-25T21:05:36.164820+0000","last_fullsized":"2024-04-25T21:05:36.164820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280051+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:05:36.280051+0000","last_peered":"2024-04-25T21:05:36.280051+0000","last_clean":"2024-04-25T21:05:36.280051+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:05:36.280051+0000","last_undegraded":"2024-04-25T21:05:36.280051+0000","last_fullsized":"2024-04-25T21:05:36.280051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812196+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:05:35.812196+0000","last_peered":"2024-04-25T21:05:35.812196+0000","last_clean":"2024-04-25T21:05:35.812196+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:05:35.812196+0000","last_undegraded":"2024-04-25T21:05:35.812196+0000","last_fullsized":"2024-04-25T21:05:35.812196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.820411+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:05:35.820411+0000","last_peered":"2024-04-25T21:05:35.820411+0000","last_clean":"2024-04-25T21:05:35.820411+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:05:35.820411+0000","last_undegraded":"2024-04-25T21:05:35.820411+0000","last_fullsized":"2024-04-25T21:05:35.820411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.816311+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:05:35.816311+0000","last_peered":"2024-04-25T21:05:35.816311+0000","last_clean":"2024-04-25T21:05:35.816311+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:05:35.816311+0000","last_undegraded":"2024-04-25T21:05:35.816311+0000","last_fullsized":"2024-04-25T21:05:35.816311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164359+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:05:36.164359+0000","last_peered":"2024-04-25T21:05:36.164359+0000","last_clean":"2024-04-25T21:05:36.164359+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:05:36.164359+0000","last_undegraded":"2024-04-25T21:05:36.164359+0000","last_fullsized":"2024-04-25T21:05:36.164359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813860+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:05:35.813860+0000","last_peered":"2024-04-25T21:05:35.813860+0000","last_clean":"2024-04-25T21:05:35.813860+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:05:35.813860+0000","last_undegraded":"2024-04-25T21:05:35.813860+0000","last_fullsized":"2024-04-25T21:05:35.813860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812840+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:05:35.812840+0000","last_peered":"2024-04-25T21:05:35.812840+0000","last_clean":"2024-04-25T21:05:35.812840+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:05:35.812840+0000","last_undegraded":"2024-04-25T21:05:35.812840+0000","last_fullsized":"2024-04-25T21:05:35.812840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815193+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:05:35.815193+0000","last_peered":"2024-04-25T21:05:35.815193+0000","last_clean":"2024-04-25T21:05:35.815193+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:05:35.815193+0000","last_undegraded":"2024-04-25T21:05:35.815193+0000","last_fullsized":"2024-04-25T21:05:35.815193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810529+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:05:35.810529+0000","last_peered":"2024-04-25T21:05:35.810529+0000","last_clean":"2024-04-25T21:05:35.810529+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:05:35.810529+0000","last_undegraded":"2024-04-25T21:05:35.810529+0000","last_fullsized":"2024-04-25T21:05:35.810529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:43.291730+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:05:43.291730+0000","last_peered":"2024-04-25T21:05:43.291730+0000","last_clean":"2024-04-25T21:05:43.291730+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:05:43.291730+0000","last_undegraded":"2024-04-25T21:05:43.291730+0000","last_fullsized":"2024-04-25T21:05:43.291730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810542+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:05:35.810542+0000","last_peered":"2024-04-25T21:05:35.810542+0000","last_clean":"2024-04-25T21:05:35.810542+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:05:35.810542+0000","last_undegraded":"2024-04-25T21:05:35.810542+0000","last_fullsized":"2024-04-25T21:05:35.810542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810498+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:05:35.810498+0000","last_peered":"2024-04-25T21:05:35.810498+0000","last_clean":"2024-04-25T21:05:35.810498+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:05:35.810498+0000","last_undegraded":"2024-04-25T21:05:35.810498+0000","last_fullsized":"2024-04-25T21:05:35.810498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810354+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:05:35.810354+0000","last_peered":"2024-04-25T21:05:35.810354+0000","last_clean":"2024-04-25T21:05:35.810354+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:05:35.810354+0000","last_undegraded":"2024-04-25T21:05:35.810354+0000","last_fullsized":"2024-04-25T21:05:35.810354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809534+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:05:35.809534+0000","last_peered":"2024-04-25T21:05:35.809534+0000","last_clean":"2024-04-25T21:05:35.809534+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:05:35.809534+0000","last_undegraded":"2024-04-25T21:05:35.809534+0000","last_fullsized":"2024-04-25T21:05:35.809534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810657+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:05:35.810657+0000","last_peered":"2024-04-25T21:05:35.810657+0000","last_clean":"2024-04-25T21:05:35.810657+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:05:35.810657+0000","last_undegraded":"2024-04-25T21:05:35.810657+0000","last_fullsized":"2024-04-25T21:05:35.810657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848543+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:05:35.848543+0000","last_peered":"2024-04-25T21:05:35.848543+0000","last_clean":"2024-04-25T21:05:35.848543+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:05:35.848543+0000","last_undegraded":"2024-04-25T21:05:35.848543+0000","last_fullsized":"2024-04-25T21:05:35.848543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.814755+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:05:35.814755+0000","last_peered":"2024-04-25T21:05:35.814755+0000","last_clean":"2024-04-25T21:05:35.814755+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:05:35.814755+0000","last_undegraded":"2024-04-25T21:05:35.814755+0000","last_fullsized":"2024-04-25T21:05:35.814755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812713+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:05:35.812713+0000","last_peered":"2024-04-25T21:05:35.812713+0000","last_clean":"2024-04-25T21:05:35.812713+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:05:35.812713+0000","last_undegraded":"2024-04-25T21:05:35.812713+0000","last_fullsized":"2024-04-25T21:05:35.812713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811944+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:05:35.811944+0000","last_peered":"2024-04-25T21:05:35.811944+0000","last_clean":"2024-04-25T21:05:35.811944+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:05:35.811944+0000","last_undegraded":"2024-04-25T21:05:35.811944+0000","last_fullsized":"2024-04-25T21:05:35.811944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809423+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:05:35.809423+0000","last_peered":"2024-04-25T21:05:35.809423+0000","last_clean":"2024-04-25T21:05:35.809423+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:05:35.809423+0000","last_undegraded":"2024-04-25T21:05:35.809423+0000","last_fullsized":"2024-04-25T21:05:35.809423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848642+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:05:35.848642+0000","last_peered":"2024-04-25T21:05:35.848642+0000","last_clean":"2024-04-25T21:05:35.848642+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:05:35.848642+0000","last_undegraded":"2024-04-25T21:05:35.848642+0000","last_fullsized":"2024-04-25T21:05:35.848642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812987+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:05:35.812987+0000","last_peered":"2024-04-25T21:05:35.812987+0000","last_clean":"2024-04-25T21:05:35.812987+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:05:35.812987+0000","last_undegraded":"2024-04-25T21:05:35.812987+0000","last_fullsized":"2024-04-25T21:05:35.812987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164713+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:05:36.164713+0000","last_peered":"2024-04-25T21:05:36.164713+0000","last_clean":"2024-04-25T21:05:36.164713+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:05:36.164713+0000","last_undegraded":"2024-04-25T21:05:36.164713+0000","last_fullsized":"2024-04-25T21:05:36.164713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810024+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:05:35.810024+0000","last_peered":"2024-04-25T21:05:35.810024+0000","last_clean":"2024-04-25T21:05:35.810024+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:05:35.810024+0000","last_undegraded":"2024-04-25T21:05:35.810024+0000","last_fullsized":"2024-04-25T21:05:35.810024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164285+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:05:36.164285+0000","last_peered":"2024-04-25T21:05:36.164285+0000","last_clean":"2024-04-25T21:05:36.164285+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:05:36.164285+0000","last_undegraded":"2024-04-25T21:05:36.164285+0000","last_fullsized":"2024-04-25T21:05:36.164285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818506+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:05:35.818506+0000","last_peered":"2024-04-25T21:05:35.818506+0000","last_clean":"2024-04-25T21:05:35.818506+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:05:35.818506+0000","last_undegraded":"2024-04-25T21:05:35.818506+0000","last_fullsized":"2024-04-25T21:05:35.818506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809406+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:05:35.809406+0000","last_peered":"2024-04-25T21:05:35.809406+0000","last_clean":"2024-04-25T21:05:35.809406+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:05:35.809406+0000","last_undegraded":"2024-04-25T21:05:35.809406+0000","last_fullsized":"2024-04-25T21:05:35.809406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813376+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:05:35.813376+0000","last_peered":"2024-04-25T21:05:35.813376+0000","last_clean":"2024-04-25T21:05:35.813376+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:05:35.813376+0000","last_undegraded":"2024-04-25T21:05:35.813376+0000","last_fullsized":"2024-04-25T21:05:35.813376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246244,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":297,"num_read_kb":217,"num_write":239,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":184713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":481464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1138688,"data_stored":1125887,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070505,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":480217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1073317,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":535418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":481868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":536420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920830,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":479999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072995,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:05:50.666 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T21:05:50.667 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T21:05:50.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T21:05:50.953 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:50.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:50 smithi037 ceph-mon[29195]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:51.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:50 smithi098 ceph-mon[32569]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:51.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:50 smithi177 ceph-mon[33043]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:05:51.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:51 smithi037 ceph-mon[29195]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:52.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:51 smithi098 ceph-mon[32569]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:51 smithi177 ceph-mon[33043]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 -- 172.21.15.37:0/4229429436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8103cd0 msgr2=0x7f37e8104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/4229429436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8103cd0 0x7f37e8104130 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f37d80099c0 tx=0x7f37d8009cd0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 -- 172.21.15.37:0/4229429436 shutdown_connections 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/4229429436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 0x7f37e810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/4229429436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8102ad0 0x7f37e8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/4229429436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8103cd0 0x7f37e8104130 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 -- 172.21.15.37:0/4229429436 >> 172.21.15.37:0/4229429436 conn(0x7f37e80fe090 msgr2=0x7f37e81004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 -- 172.21.15.37:0/4229429436 shutdown_connections 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.667+0000 7f37ef8b4700 1 -- 172.21.15.37:0/4229429436 wait complete. 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 Processor -- start 2024-04-25T21:05:52.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 -- start start 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8103cd0 0x7f37e81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 0x7f37e81a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37e810e660 con 0x7f37e8102ad0 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37e810e4e0 con 0x7f37e8104670 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ef8b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f37e810e7e0 con 0x7f37e8103cd0 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34684/0 (socket says 172.21.15.37:34684) 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 -- 172.21.15.37:0/860497589 learned_addr learned my addr 172.21.15.37:0/860497589 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ede51700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 0x7f37e81a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:52.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ece4f700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8103cd0 0x7f37e81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 -- 172.21.15.37:0/860497589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8103cd0 msgr2=0x7f37e81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8103cd0 0x7f37e81990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 -- 172.21.15.37:0/860497589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 msgr2=0x7f37e81a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 0x7f37e81a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.668+0000 7f37ed650700 1 -- 172.21.15.37:0/860497589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e81a0810 con 0x7f37e8102ad0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37ed650700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f37e400b530 tx=0x7f37e400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4015020 con 0x7f37e8102ad0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e4011ce0 con 0x7f37e8102ad0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e81a0b00 con 0x7f37e8102ad0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4010d60 con 0x7f37e8102ad0 2024-04-25T21:05:52.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.669+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a1040 con 0x7f37e8102ad0 2024-04-25T21:05:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.670+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8050f80 con 0x7f37e8102ad0 2024-04-25T21:05:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.671+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37e4018020 con 0x7f37e8102ad0 2024-04-25T21:05:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.672+0000 7f37de7fc700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 0x7f37d40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:52.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.672+0000 7f37ece4f700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 0x7f37d40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.672+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f37e40cfc40 con 0x7f37e8102ad0 2024-04-25T21:05:52.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.672+0000 7f37ece4f700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 0x7f37d40a6240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:52.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.676+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37e40d6080 con 0x7f37e8102ad0 2024-04-25T21:05:52.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.969+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f37e802e190 con 0x7f37e8102ad0 2024-04-25T21:05:52.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.971+0000 7f37de7fc700 1 -- 172.21.15.37:0/860497589 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f37e401f020 con 0x7f37e8102ad0 2024-04-25T21:05:52.972 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T21:05:52.972 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCwxSpm0QjgORAAEpjsrS0zLY1GkTDrgarmjw== 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 msgr2=0x7f37d40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 0x7f37d40a6240 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37d8000c00 tx=0x7f37d8005fb0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 msgr2=0x7f37e8198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f37e400b530 tx=0x7f37e400b8f0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 shutdown_connections 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f37d40a3d80 0x7f37d40a6240 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37e8104670 0x7f37e81a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e8102ad0 0x7f37e8198bb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 --2- 172.21.15.37:0/860497589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37e8103cd0 0x7f37e81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:52.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.973+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 >> 172.21.15.37:0/860497589 conn(0x7f37e80fe090 msgr2=0x7f37e8106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:52.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.974+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 shutdown_connections 2024-04-25T21:05:52.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:52.974+0000 7f37ef8b4700 1 -- 172.21.15.37:0/860497589 wait complete. 2024-04-25T21:05:53.618 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T21:05:53.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T21:05:53.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T21:05:53.659 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T21:05:53.659 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T21:05:53.659 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T21:05:53.961 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:54.118 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:53 smithi037 ceph-mon[29195]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T21:05:54.118 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:53 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T21:05:54.118 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:53 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T21:05:54.118 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:53 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:53 smithi098 ceph-mon[32569]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T21:05:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:53 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T21:05:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:53 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T21:05:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:53 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:54.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:53 smithi177 ceph-mon[33043]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T21:05:54.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:53 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T21:05:54.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:53 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/860497589' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T21:05:54.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:53 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:55.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 -- 172.21.15.37:0/1725013748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0101940 msgr2=0x7fade0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:55.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 --2- 172.21.15.37:0/1725013748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0101940 0x7fade0101dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009990 tx=0x7fadd4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 -- 172.21.15.37:0/1725013748 shutdown_connections 2024-04-25T21:05:55.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 --2- 172.21.15.37:0/1725013748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0102300 0x7fade010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 --2- 172.21.15.37:0/1725013748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 0x7fade0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 --2- 172.21.15.37:0/1725013748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0101940 0x7fade0101dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.651+0000 7fade65e4700 1 -- 172.21.15.37:0/1725013748 >> 172.21.15.37:0/1725013748 conn(0x7fade00788f0 msgr2=0x7fade0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:55.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.652+0000 7fade65e4700 1 -- 172.21.15.37:0/1725013748 shutdown_connections 2024-04-25T21:05:55.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.652+0000 7fade65e4700 1 -- 172.21.15.37:0/1725013748 wait complete. 2024-04-25T21:05:55.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.652+0000 7fade65e4700 1 Processor -- start 2024-04-25T21:05:55.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.652+0000 7fade65e4700 1 -- start start 2024-04-25T21:05:55.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0101940 0x7fade01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:55.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 0x7fade0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fade0111ae0 con 0x7fade0102300 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fade0111960 con 0x7fade0107990 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade65e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fade0111c60 con 0x7fade0101940 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:55.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34698/0 (socket says 172.21.15.37:34698) 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 -- 172.21.15.37:0/2623970567 learned_addr learned my addr 172.21.15.37:0/2623970567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddffff700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0101940 0x7fade01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7fade4b81700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 0x7fade0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 -- 172.21.15.37:0/2623970567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0101940 msgr2=0x7fade01034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0101940 0x7fade01034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 -- 172.21.15.37:0/2623970567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 msgr2=0x7fade0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 0x7fade0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 -- 172.21.15.37:0/2623970567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01a7660 con 0x7fade0102300 2024-04-25T21:05:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddf7fe700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.653+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4027070 con 0x7fade0102300 2024-04-25T21:05:55.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.654+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01a78f0 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.654+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd4003f20 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.654+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a7da0 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.654+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4022ca0 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.655+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade0069730 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.656+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadd402a070 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.656+0000 7faddd7fa700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 0x7fadc80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.657+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fadd40d1aa0 con 0x7fade0102300 2024-04-25T21:05:55.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.657+0000 7faddffff700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 0x7fadc80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:55.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.657+0000 7faddffff700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 0x7fadc80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005950 tx=0x7fadd000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:55.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.664+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadd4031070 con 0x7fade0102300 2024-04-25T21:05:55.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.963+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fade002df80 con 0x7fade0102300 2024-04-25T21:05:55.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.968+0000 7faddd7fa700 1 -- 172.21.15.37:0/2623970567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264996 (secure 0 0 0) 0x7fadd4013070 con 0x7fade0102300 2024-04-25T21:05:55.972 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 msgr2=0x7fadc80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 0x7fadc80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fadd0005950 tx=0x7fadd000b410 comp rx=0 tx=0).stop 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 msgr2=0x7fade0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fadd4000c00 tx=0x7fadd40047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 shutdown_connections 2024-04-25T21:05:55.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fadc80a3cd0 0x7fadc80a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fade0107990 0x7fade0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fade0101940 0x7fade01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 --2- 172.21.15.37:0/2623970567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade0102300 0x7fade0103a30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 >> 172.21.15.37:0/2623970567 conn(0x7fade00788f0 msgr2=0x7fade006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 shutdown_connections 2024-04-25T21:05:55.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:55.971+0000 7fade65e4700 1 -- 172.21.15.37:0/2623970567 wait complete. 2024-04-25T21:05:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:55 smithi098 ceph-mon[32569]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T21:05:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:55 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:55 smithi037 ceph-mon[29195]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T21:05:56.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:55 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:55 smithi177 ceph-mon[33043]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-04-25T21:05:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:55 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:57.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:56 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2623970567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T21:05:57.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:56 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2623970567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T21:05:57.261 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":2407888079},{"type":"v1","addr":"172.21.15.37:6801","nonce":2407888079}]},"active_addr":"172.21.15.37:6801/2407888079","active_change":"2024-04-25T20:57:34.093056+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1398754476}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":234140491}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3727603400}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1634118111}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":790728294}]}]} 2024-04-25T21:05:57.267 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T21:05:57.267 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T21:05:57.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:05:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:56 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2623970567' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T21:05:57.548 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:05:57.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:57 smithi037 ceph-mon[29195]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T21:05:57.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:57.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:57 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:58.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:57 smithi098 ceph-mon[32569]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T21:05:58.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:57 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:58.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:57 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:58.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:57 smithi177 ceph-mon[33043]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-25T21:05:58.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:57 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:58.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:57 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:58.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:58.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:58.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:59.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:59.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:59.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:59.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- 172.21.15.37:0/1324644011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4102ac0 msgr2=0x7f10d4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 --2- 172.21.15.37:0/1324644011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4102ac0 0x7f10d4102f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009990 tx=0x7f10c8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- 172.21.15.37:0/1324644011 shutdown_connections 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 --2- 172.21.15.37:0/1324644011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4103480 0x7f10d410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 --2- 172.21.15.37:0/1324644011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4108a70 0x7f10d4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 --2- 172.21.15.37:0/1324644011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4102ac0 0x7f10d4102f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- 172.21.15.37:0/1324644011 >> 172.21.15.37:0/1324644011 conn(0x7f10d4079bb0 msgr2=0x7f10d4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- 172.21.15.37:0/1324644011 shutdown_connections 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- 172.21.15.37:0/1324644011 wait complete. 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 Processor -- start 2024-04-25T21:05:59.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.280+0000 7f10d953f700 1 -- start start 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 0x7f10d4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4108a70 0x7f10d419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10d4112d00 con 0x7f10d4103480 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10d4112b80 con 0x7f10d4108a70 2024-04-25T21:05:59.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d953f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f10d4112e80 con 0x7f10d4102ac0 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51242/0 (socket says 172.21.15.37:51242) 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 -- 172.21.15.37:0/459916905 learned_addr learned my addr 172.21.15.37:0/459916905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d37fe700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4108a70 0x7f10d419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d2ffd700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 0x7f10d4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 -- 172.21.15.37:0/459916905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 msgr2=0x7f10d4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 0x7f10d4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 -- 172.21.15.37:0/459916905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4108a70 msgr2=0x7f10d419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4108a70 0x7f10d419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 -- 172.21.15.37:0/459916905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d419e310 con 0x7f10d4103480 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d2ffd700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 0x7f10d4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10d27fc700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8027070 con 0x7f10d4103480 2024-04-25T21:05:59.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.281+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c80041d0 con 0x7f10d4103480 2024-04-25T21:05:59.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.282+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d419e5a0 con 0x7f10d4103480 2024-04-25T21:05:59.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.282+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c800f460 con 0x7f10d4103480 2024-04-25T21:05:59.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.282+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d41a5e80 con 0x7f10d4103480 2024-04-25T21:05:59.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.283+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d41041a0 con 0x7f10d4103480 2024-04-25T21:05:59.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.284+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f10c802a070 con 0x7f10d4103480 2024-04-25T21:05:59.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.284+0000 7f10bbfff700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 0x7f10bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:05:59.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.285+0000 7f10d2ffd700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 0x7f10bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:05:59.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.285+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f10c80d19c0 con 0x7f10d4103480 2024-04-25T21:05:59.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.285+0000 7f10d2ffd700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 0x7f10bc0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10c4007900 tx=0x7f10c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:05:59.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.288+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10c8031070 con 0x7f10d4103480 2024-04-25T21:05:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:05:59.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.499+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10d402df80 con 0x7f10d4103480 2024-04-25T21:05:59.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.501+0000 7f10bbfff700 1 -- 172.21.15.37:0/459916905 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22571 (secure 0 0 0) 0x7f10c8013070 con 0x7f10d4103480 2024-04-25T21:05:59.502 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:05:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:35.806342+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:05:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 msgr2=0x7f10bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:59.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 0x7f10bc0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f10c4007900 tx=0x7f10c4008040 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 msgr2=0x7f10d4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f10c8000c00 tx=0x7f10c80047b0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 shutdown_connections 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f10bc0a3cd0 0x7f10bc0a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10d4108a70 0x7f10d419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f10d4102ac0 0x7f10d4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 --2- 172.21.15.37:0/459916905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10d4103480 0x7f10d4199840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 >> 172.21.15.37:0/459916905 conn(0x7f10d4079bb0 msgr2=0x7f10d4101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 shutdown_connections 2024-04-25T21:05:59.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:05:59.503+0000 7f10d953f700 1 -- 172.21.15.37:0/459916905 wait complete. 2024-04-25T21:06:00.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:59 smithi037 ceph-mon[29195]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:00.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:05:59 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/459916905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:00.120 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T21:06:00.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:06:00.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:59 smithi098 ceph-mon[32569]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:00.173 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:05:59 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/459916905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:00.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:59 smithi177 ceph-mon[33043]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:00.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:05:59 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/459916905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:00.374 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:01.973 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:01 smithi037 ceph-mon[29195]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:02.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:01 smithi098 ceph-mon[32569]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 -- 172.21.15.37:0/1296554765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 msgr2=0x7f56b006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 --2- 172.21.15.37:0/1296554765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b006b440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009990 tx=0x7f56a4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 -- 172.21.15.37:0/1296554765 shutdown_connections 2024-04-25T21:06:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 --2- 172.21.15.37:0/1296554765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 0x7f56b010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 --2- 172.21.15.37:0/1296554765 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 0x7f56b0108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 --2- 172.21.15.37:0/1296554765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b006b440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 -- 172.21.15.37:0/1296554765 >> 172.21.15.37:0/1296554765 conn(0x7f56b00788b0 msgr2=0x7f56b0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 -- 172.21.15.37:0/1296554765 shutdown_connections 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.209+0000 7f56b633d700 1 -- 172.21.15.37:0/1296554765 wait complete. 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 Processor -- start 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 -- start start 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 0x7f56b0101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 0x7f56b0102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0112c10 con 0x7f56b006afe0 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0112a90 con 0x7f56b006b980 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b633d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0112d90 con 0x7f56b0108a20 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51272/0 (socket says 172.21.15.37:51272) 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 -- 172.21.15.37:0/3092634047 learned_addr learned my addr 172.21.15.37:0/3092634047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56b48da700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 0x7f56b0102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56af7fe700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 0x7f56b0101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 -- 172.21.15.37:0/3092634047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 msgr2=0x7f56b0102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 0x7f56b0102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 -- 172.21.15.37:0/3092634047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 msgr2=0x7f56b0101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 0x7f56b0101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.210+0000 7f56affff700 1 -- 172.21.15.37:0/3092634047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b01a7570 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56affff700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c9b0 tx=0x7f56a000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007b00 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b01a7860 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56a0007ca0 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0018730 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.211+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b01a7df0 con 0x7f56b006afe0 2024-04-25T21:06:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.212+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b0069730 con 0x7f56b006afe0 2024-04-25T21:06:02.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.213+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56a000f450 con 0x7f56b006afe0 2024-04-25T21:06:02.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.214+0000 7f56ad7fa700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 0x7f56980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:02.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.214+0000 7f56af7fe700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 0x7f56980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.214+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f56a00d0570 con 0x7f56b006afe0 2024-04-25T21:06:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.214+0000 7f56af7fe700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 0x7f56980a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b5c0 tx=0x7f56a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.219+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56a0014030 con 0x7f56b006afe0 2024-04-25T21:06:02.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:01 smithi177 ceph-mon[33043]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-25T21:06:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.430+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56b002df80 con 0x7f56b006afe0 2024-04-25T21:06:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.432+0000 7f56ad7fa700 1 -- 172.21.15.37:0/3092634047 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22571 (secure 0 0 0) 0x7f56a009ce00 con 0x7f56b006afe0 2024-04-25T21:06:02.433 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:06:02.433 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:35.806342+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 msgr2=0x7f56980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 0x7f56980a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b5c0 tx=0x7f56a401a040 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 msgr2=0x7f56b0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c9b0 tx=0x7f56a000cd70 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 shutdown_connections 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56b0108a20 0x7f56b0102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56b006afe0 0x7f56b0101a00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56980a3cd0 0x7f56980a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 --2- 172.21.15.37:0/3092634047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b006b980 0x7f56b0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.434+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 >> 172.21.15.37:0/3092634047 conn(0x7f56b00788b0 msgr2=0x7f56b00ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:02.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.435+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 shutdown_connections 2024-04-25T21:06:02.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:02.435+0000 7f56b633d700 1 -- 172.21.15.37:0/3092634047 wait complete. 2024-04-25T21:06:03.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:02 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3092634047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:03.168 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:02 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3092634047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:03.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T21:06:03.201 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T21:06:03.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T21:06:03.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T21:06:03.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T21:06:03.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T21:06:03.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T21:06:03.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T21:06:03.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T21:06:03.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T21:06:03.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T21:06:03.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T21:06:03.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:02 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3092634047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:06:04.002 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.003 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.006 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.013 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.024 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:04.154 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:03 smithi037 ceph-mon[29195]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T21:06:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:03 smithi098 ceph-mon[32569]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T21:06:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:03 smithi177 ceph-mon[33043]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-04-25T21:06:06.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:05 smithi037 ceph-mon[29195]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:05 smithi098 ceph-mon[32569]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:06.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:05 smithi177 ceph-mon[33043]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:08.171 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:07 smithi098 ceph-mon[32569]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:08.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:07 smithi037 ceph-mon[29195]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:08.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:07 smithi177 ceph-mon[33043]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:10.137 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:09 smithi037 ceph-mon[29195]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:10.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:09 smithi098 ceph-mon[32569]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:10.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:09 smithi177 ceph-mon[33043]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:11 smithi037 ceph-mon[29195]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:11 smithi098 ceph-mon[32569]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:12.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:11 smithi177 ceph-mon[33043]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:13 smithi098 ceph-mon[32569]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:14.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:13 smithi037 ceph-mon[29195]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:14.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:13 smithi177 ceph-mon[33043]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:15.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3370594437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c101960 msgr2=0x7f7a8c101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3370594437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c101960 0x7f7a8c101de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009990 tx=0x7f7a74009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3370594437 shutdown_connections 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3370594437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a8c102320 0x7f7a8c10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3370594437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c107960 0x7f7a8c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3370594437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c101960 0x7f7a8c101de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3370594437 >> 172.21.15.37:0/3370594437 conn(0x7f7a8c0789e0 msgr2=0x7f7a8c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3370594437 shutdown_connections 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3370594437 wait complete. 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.745+0000 7f7a91bb0700 1 Processor -- start 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 -- start start 2024-04-25T21:06:15.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c101960 0x7f7a8c0fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a8c102320 0x7f7a8c0feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 0x7f7a8c0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111b70 con 0x7f7a8c107960 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c1119f0 con 0x7f7a8c101960 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a91bb0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c111cf0 con 0x7f7a8c102320 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 0x7f7a8c0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 0x7f7a8c0ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46756/0 (socket says 172.21.15.37:46756) 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 -- 172.21.15.37:0/3153550694 learned_addr learned my addr 172.21.15.37:0/3153550694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8b7fe700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c101960 0x7f7a8c0fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8affd700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a8c102320 0x7f7a8c0feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 -- 172.21.15.37:0/3153550694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a8c102320 msgr2=0x7f7a8c0feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a8c102320 0x7f7a8c0feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 -- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c101960 msgr2=0x7f7a8c0fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a8c101960 0x7f7a8c0fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a7630 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.746+0000 7f7a8bfff700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 0x7f7a8c0ff410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007ee0 tx=0x7f7a8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.747+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84004bb0 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.747+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a7920 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.747+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a84004d50 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.747+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a84005920 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.747+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a7e80 con 0x7f7a8c107960 2024-04-25T21:06:15.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.748+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a8c0630d0 con 0x7f7a8c107960 2024-04-25T21:06:15.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.749+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7a8401a020 con 0x7f7a8c107960 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.749+0000 7f7a88ff9700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7a780a3cd0 0x7f7a780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a8b7fe700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7a780a3cd0 0x7f7a780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f7a840d02c0 con 0x7f7a8c107960 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a88ff9700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] conn(0x7f7a780ad810 0x7f7a780afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a780b0380 con 0x7f7a780ad810 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7a840d0710 con 0x7f7a8c107960 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a8b7fe700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7a780a3cd0 0x7f7a780a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005950 tx=0x7f7a7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.750+0000 7f7a8affd700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] conn(0x7f7a780ad810 0x7f7a780afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.751+0000 7f7a8affd700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] conn(0x7f7a780ad810 0x7f7a780afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.752+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== osd.10 v2:172.21.15.98:6824/592671509 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a780b0380 con 0x7f7a780ad810 2024-04-25T21:06:15.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.768+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 --> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a8c063550 con 0x7f7a780ad810 2024-04-25T21:06:15.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.768+0000 7f7a88ff9700 1 -- 172.21.15.37:0/3153550694 <== osd.10 v2:172.21.15.98:6824/592671509 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a8c063550 con 0x7f7a780ad810 2024-04-25T21:06:15.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] conn(0x7f7a780ad810 msgr2=0x7f7a780afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] conn(0x7f7a780ad810 0x7f7a780afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7a780a3cd0 msgr2=0x7f7a780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f7a780a3cd0 0x7f7a780a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005950 tx=0x7f7a7c00b410 comp rx=0 tx=0).stop 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 msgr2=0x7f7a8c0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 --2- 172.21.15.37:0/3153550694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a8c107960 0x7f7a8c0ff410 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7a84007ee0 tx=0x7f7a8400d3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a8bfff700 1 -- 172.21.15.37:0/3153550694 reap_dead start 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 shutdown_connections 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 >> 172.21.15.37:0/3153550694 conn(0x7f7a8c0789e0 msgr2=0x7f7a8c06b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 shutdown_connections 2024-04-25T21:06:15.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.769+0000 7f7a91bb0700 1 -- 172.21.15.37:0/3153550694 wait complete. 2024-04-25T21:06:15.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- 172.21.15.37:0/1259902474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40101940 msgr2=0x7f9f40101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 --2- 172.21.15.37:0/1259902474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40101940 0x7f9f40101dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009990 tx=0x7f9f30009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- 172.21.15.37:0/1259902474 shutdown_connections 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 --2- 172.21.15.37:0/1259902474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f40102300 0x7f9f4010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 --2- 172.21.15.37:0/1259902474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f40107990 0x7f9f40107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 --2- 172.21.15.37:0/1259902474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40101940 0x7f9f40101dc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- 172.21.15.37:0/1259902474 >> 172.21.15.37:0/1259902474 conn(0x7f9f400788f0 msgr2=0x7f9f40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- 172.21.15.37:0/1259902474 shutdown_connections 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- 172.21.15.37:0/1259902474 wait complete. 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 Processor -- start 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.946+0000 7f9f477bb700 1 -- start start 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f40101940 0x7f9f40199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 0x7f9f4019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f40107990 0x7f9f401a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40111ae0 con 0x7f9f40102300 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40111960 con 0x7f9f40101940 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f40111c60 con 0x7f9f40107990 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 0x7f9f4019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 0x7f9f4019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46778/0 (socket says 172.21.15.37:46778) 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 -- 172.21.15.37:0/3198528579 learned_addr learned my addr 172.21.15.37:0/3198528579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f45557700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f40101940 0x7f9f40199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f45d58700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f40107990 0x7f9f401a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 -- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f40107990 msgr2=0x7f9f401a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f40107990 0x7f9f401a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 -- 172.21.15.37:0/3198528579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f40101940 msgr2=0x7f9f40199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f40101940 0x7f9f40199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f401a5500 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f44d56700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 0x7f9f4019a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f300057f0 tx=0x7f9f30003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30027070 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f30004020 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f401a56d0 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30022cf0 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.947+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f401a5bb0 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.948+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f40063110 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.950+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f3002a070 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.950+0000 7f9f367fc700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f2c0a3cd0 0x7f9f2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.950+0000 7f9f45557700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f2c0a3cd0 0x7f9f2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.950+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f9f300d1ef0 con 0x7f9f40102300 2024-04-25T21:06:15.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f367fc700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] conn(0x7f9f2c0ad810 0x7f9f2c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f2c0b0380 con 0x7f9f2c0ad810 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f9f300d2340 con 0x7f9f40102300 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f45557700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f2c0a3cd0 0x7f9f2c0a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00abd0 tx=0x7f9f3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f45d58700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] conn(0x7f9f2c0ad810 0x7f9f2c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.951+0000 7f9f45d58700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] conn(0x7f9f2c0ad810 0x7f9f2c0afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:15.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.952+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== osd.2 v2:172.21.15.177:6800/5263269 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f2c0b0380 con 0x7f9f2c0ad810 2024-04-25T21:06:15.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.971+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 --> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f40063510 con 0x7f9f2c0ad810 2024-04-25T21:06:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.971+0000 7f9f367fc700 1 -- 172.21.15.37:0/3198528579 <== osd.2 v2:172.21.15.177:6800/5263269 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f40063510 con 0x7f9f2c0ad810 2024-04-25T21:06:15.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] conn(0x7f9f2c0ad810 msgr2=0x7f9f2c0afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] conn(0x7f9f2c0ad810 0x7f9f2c0afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f2c0a3cd0 msgr2=0x7f9f2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f2c0a3cd0 0x7f9f2c0a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00abd0 tx=0x7f9f3c00a380 comp rx=0 tx=0).stop 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 msgr2=0x7f9f4019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 --2- 172.21.15.37:0/3198528579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f40102300 0x7f9f4019a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f300057f0 tx=0x7f9f30003680 comp rx=0 tx=0).stop 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f45d58700 1 -- 172.21.15.37:0/3198528579 reap_dead start 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 shutdown_connections 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 >> 172.21.15.37:0/3198528579 conn(0x7f9f400788f0 msgr2=0x7f9f400ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 shutdown_connections 2024-04-25T21:06:15.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:15.972+0000 7f9f477bb700 1 -- 172.21.15.37:0/3198528579 wait complete. 2024-04-25T21:06:16.157 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:15 smithi177 ceph-mon[33043]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:16.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:15 smithi098 ceph-mon[32569]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:16.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:15 smithi037 ceph-mon[29195]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 -- 172.21.15.37:0/4010162282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88107990 msgr2=0x7fea88107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 --2- 172.21.15.37:0/4010162282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88107990 0x7fea88107d70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fea78009990 tx=0x7fea78009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 -- 172.21.15.37:0/4010162282 shutdown_connections 2024-04-25T21:06:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 --2- 172.21.15.37:0/4010162282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea88102300 0x7fea8810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 --2- 172.21.15.37:0/4010162282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88107990 0x7fea88107d70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 --2- 172.21.15.37:0/4010162282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88101940 0x7fea88101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 -- 172.21.15.37:0/4010162282 >> 172.21.15.37:0/4010162282 conn(0x7fea880788f0 msgr2=0x7fea88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 -- 172.21.15.37:0/4010162282 shutdown_connections 2024-04-25T21:06:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.324+0000 7fea8cb89700 1 -- 172.21.15.37:0/4010162282 wait complete. 2024-04-25T21:06:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 Processor -- start 2024-04-25T21:06:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 -- start start 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea88101940 0x7fea88197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 0x7fea88198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88107990 0x7fea881987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea88111b00 con 0x7fea88102300 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea88111980 con 0x7fea88107990 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.325+0000 7fea8cb89700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea88111c80 con 0x7fea88101940 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 0x7fea88198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 0x7fea88198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46792/0 (socket says 172.21.15.37:46792) 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 -- 172.21.15.37:0/166893473 learned_addr learned my addr 172.21.15.37:0/166893473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea86d9d700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88107990 0x7fea881987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 -- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea88101940 msgr2=0x7fea88197d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea88101940 0x7fea88197d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 -- 172.21.15.37:0/166893473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88107990 msgr2=0x7fea881987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea88107990 0x7fea881987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea881a7540 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea85d9b700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 0x7fea88198270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c9b0 tx=0x7fea7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c007b00 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881a7830 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c007ca0 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.326+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c018660 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.327+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a7dc0 con 0x7fea88102300 2024-04-25T21:06:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.327+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea88063190 con 0x7fea88102300 2024-04-25T21:06:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.330+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea7c00f450 con 0x7fea88102300 2024-04-25T21:06:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea777fe700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fea700a3dd0 0x7fea700a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fea7c0d0970 con 0x7fea88102300 2024-04-25T21:06:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea777fe700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] conn(0x7fea700ad910 0x7fea700afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea7c014ec0 con 0x7fea700ad910 2024-04-25T21:06:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fea7c0d0d60 con 0x7fea88102300 2024-04-25T21:06:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.331+0000 7fea8659c700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fea700a3dd0 0x7fea700a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.332+0000 7fea86d9d700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] conn(0x7fea700ad910 0x7fea700afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.332+0000 7fea8659c700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fea700a3dd0 0x7fea700a6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea78005b40 tx=0x7fea78005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.332+0000 7fea86d9d700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] conn(0x7fea700ad910 0x7fea700afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.333+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== osd.11 v2:172.21.15.177:6824/2245259996 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea7c014ec0 con 0x7fea700ad910 2024-04-25T21:06:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.351+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 --> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea88063590 con 0x7fea700ad910 2024-04-25T21:06:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.351+0000 7fea777fe700 1 -- 172.21.15.37:0/166893473 <== osd.11 v2:172.21.15.177:6824/2245259996 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea88063590 con 0x7fea700ad910 2024-04-25T21:06:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] conn(0x7fea700ad910 msgr2=0x7fea700afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] conn(0x7fea700ad910 0x7fea700afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fea700a3dd0 msgr2=0x7fea700a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fea700a3dd0 0x7fea700a6290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fea78005b40 tx=0x7fea78005a90 comp rx=0 tx=0).stop 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 msgr2=0x7fea88198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 --2- 172.21.15.37:0/166893473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea88102300 0x7fea88198270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c9b0 tx=0x7fea7c00ccc0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea86d9d700 1 -- 172.21.15.37:0/166893473 reap_dead start 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 shutdown_connections 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 >> 172.21.15.37:0/166893473 conn(0x7fea880788f0 msgr2=0x7fea881006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 shutdown_connections 2024-04-25T21:06:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.352+0000 7fea8cb89700 1 -- 172.21.15.37:0/166893473 wait complete. 2024-04-25T21:06:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 -- 172.21.15.37:0/2690962616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 msgr2=0x7f2608108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 --2- 172.21.15.37:0/2690962616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f2608108e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25f800b0a0 tx=0x7f25f800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 -- 172.21.15.37:0/2690962616 shutdown_connections 2024-04-25T21:06:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 --2- 172.21.15.37:0/2690962616 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 0x7f260810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 --2- 172.21.15.37:0/2690962616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f2608108e40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 --2- 172.21.15.37:0/2690962616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 0x7f260806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 -- 172.21.15.37:0/2690962616 >> 172.21.15.37:0/2690962616 conn(0x7f26080789e0 msgr2=0x7f2608078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 -- 172.21.15.37:0/2690962616 shutdown_connections 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.940+0000 7f260c99a700 1 -- 172.21.15.37:0/2690962616 wait complete. 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 Processor -- start 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 -- start start 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 0x7f260819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 0x7f260819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f260819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2608112c40 con 0x7f2608108a60 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2608112ac0 con 0x7f260806b1f0 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260c99a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2608112dc0 con 0x7f260806bbb0 2024-04-25T21:06:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f260819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f260819fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46798/0 (socket says 172.21.15.37:46798) 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 -- 172.21.15.37:0/2443687579 learned_addr learned my addr 172.21.15.37:0/2443687579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2605d9b700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 0x7f260819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260659c700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 0x7f260819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 -- 172.21.15.37:0/2443687579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 msgr2=0x7f260819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 0x7f260819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 -- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 msgr2=0x7f260819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 0x7f260819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2606d9d700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081a0320 con 0x7f2608108a60 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f2605d9b700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f260806bbb0 0x7f260819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.941+0000 7f260659c700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f260806b1f0 0x7f260819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f2606d9d700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f260819fbe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f260000c900 tx=0x7f260000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600007b00 con 0x7f2608108a60 2024-04-25T21:06:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2600007ca0 con 0x7f2608108a60 2024-04-25T21:06:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26081a0610 con 0x7f2608108a60 2024-04-25T21:06:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600018660 con 0x7f2608108a60 2024-04-25T21:06:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.942+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081a7e90 con 0x7f2608108a60 2024-04-25T21:06:17.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.943+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f26080630d0 con 0x7f2608108a60 2024-04-25T21:06:17.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.944+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f260000f450 con 0x7f2608108a60 2024-04-25T21:06:17.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f25f40a3cd0 0x7f25f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f260659c700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f25f40a3cd0 0x7f25f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f26000d0670 con 0x7f2608108a60 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f25ff7fe700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] conn(0x7f25f40ad810 0x7f25f40afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2600010e50 con 0x7f25f40ad810 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f26000d0a20 con 0x7f2608108a60 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f260659c700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f25f40a3cd0 0x7f25f40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009fd0 tx=0x7f25f800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.945+0000 7f2605d9b700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] conn(0x7f25f40ad810 0x7f25f40afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.946+0000 7f2605d9b700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] conn(0x7f25f40ad810 0x7f25f40afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.947+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== osd.4 v2:172.21.15.98:6808/2300793190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2600010e50 con 0x7f25f40ad810 2024-04-25T21:06:17.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.963+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 --> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2608063550 con 0x7f25f40ad810 2024-04-25T21:06:17.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.963+0000 7f25ff7fe700 1 -- 172.21.15.37:0/2443687579 <== osd.4 v2:172.21.15.98:6808/2300793190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2608063550 con 0x7f25f40ad810 2024-04-25T21:06:17.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.963+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] conn(0x7f25f40ad810 msgr2=0x7f25f40afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.963+0000 7f260c99a700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] conn(0x7f25f40ad810 0x7f25f40afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f25f40a3cd0 msgr2=0x7f25f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f25f40a3cd0 0x7f25f40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009fd0 tx=0x7f25f800b7b0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 msgr2=0x7f260819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 --2- 172.21.15.37:0/2443687579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2608108a60 0x7f260819fbe0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f260000c900 tx=0x7f260000ccc0 comp rx=0 tx=0).stop 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f2606d9d700 1 -- 172.21.15.37:0/2443687579 reap_dead start 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 shutdown_connections 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 >> 172.21.15.37:0/2443687579 conn(0x7f26080789e0 msgr2=0x7f260810ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 shutdown_connections 2024-04-25T21:06:17.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:17.964+0000 7f260c99a700 1 -- 172.21.15.37:0/2443687579 wait complete. 2024-04-25T21:06:17.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:17 smithi037 ceph-mon[29195]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:17 smithi098 ceph-mon[32569]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:18.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:17 smithi177 ceph-mon[33043]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:18.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.872+0000 7f953cc9e700 1 -- 172.21.15.37:0/3893711658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9538107c10 msgr2=0x7f9538107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.872+0000 7f953cc9e700 1 --2- 172.21.15.37:0/3893711658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9538107c10 0x7f9538107ff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95280099c0 tx=0x7f9528009cd0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 -- 172.21.15.37:0/3893711658 shutdown_connections 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 --2- 172.21.15.37:0/3893711658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9538102eb0 0x7f953810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 --2- 172.21.15.37:0/3893711658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9538107c10 0x7f9538107ff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 --2- 172.21.15.37:0/3893711658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95381024f0 0x7f9538102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 -- 172.21.15.37:0/3893711658 >> 172.21.15.37:0/3893711658 conn(0x7f9538076c60 msgr2=0x7f9538077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 -- 172.21.15.37:0/3893711658 shutdown_connections 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 -- 172.21.15.37:0/3893711658 wait complete. 2024-04-25T21:06:18.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 Processor -- start 2024-04-25T21:06:18.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.873+0000 7f953cc9e700 1 -- start start 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 0x7f9538199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9538102eb0 0x7f95381996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9538107c10 0x7f953819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111c30 con 0x7f95381024f0 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111ab0 con 0x7f9538102eb0 2024-04-25T21:06:18.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953cc9e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111db0 con 0x7f9538107c10 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 0x7f9538199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 0x7f9538199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59068/0 (socket says 172.21.15.37:59068) 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 -- 172.21.15.37:0/1167301661 learned_addr learned my addr 172.21.15.37:0/1167301661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f9535d9b700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9538102eb0 0x7f95381996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f9536d9d700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9538107c10 0x7f953819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 -- 172.21.15.37:0/1167301661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9538107c10 msgr2=0x7f953819d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9538107c10 0x7f953819d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 -- 172.21.15.37:0/1167301661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9538102eb0 msgr2=0x7f95381996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9538102eb0 0x7f95381996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953819e130 con 0x7f95381024f0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.874+0000 7f953659c700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 0x7f9538199170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95280098e0 tx=0x7f95280057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.875+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528027070 con 0x7f95381024f0 2024-04-25T21:06:18.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.875+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953819e3c0 con 0x7f95381024f0 2024-04-25T21:06:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.875+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9528003f20 con 0x7f95381024f0 2024-04-25T21:06:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.875+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528022da0 con 0x7f95381024f0 2024-04-25T21:06:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.875+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a5ca0 con 0x7f95381024f0 2024-04-25T21:06:18.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.876+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9538063150 con 0x7f95381024f0 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.877+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f952802a070 con 0x7f95381024f0 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.877+0000 7f95277fe700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f95200a3cd0 0x7f95200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.877+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f95280d1da0 con 0x7f95381024f0 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.877+0000 7f9535d9b700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f95200a3cd0 0x7f95200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f95277fe700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] conn(0x7f95200ad810 0x7f95200afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9528027e70 con 0x7f95200ad810 2024-04-25T21:06:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f95280d21f0 con 0x7f95381024f0 2024-04-25T21:06:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f9536d9d700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] conn(0x7f95200ad810 0x7f95200afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f9535d9b700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f95200a3cd0 0x7f95200a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95380772d0 tx=0x7f952c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.878+0000 7f9536d9d700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] conn(0x7f95200ad810 0x7f95200afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:18.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.879+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== osd.9 v2:172.21.15.37:6826/884722198 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9528027e70 con 0x7f95200ad810 2024-04-25T21:06:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.895+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 --> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9538063550 con 0x7f95200ad810 2024-04-25T21:06:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.895+0000 7f95277fe700 1 -- 172.21.15.37:0/1167301661 <== osd.9 v2:172.21.15.37:6826/884722198 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9538063550 con 0x7f95200ad810 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] conn(0x7f95200ad810 msgr2=0x7f95200afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] conn(0x7f95200ad810 0x7f95200afcb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f95200a3cd0 msgr2=0x7f95200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f95200a3cd0 0x7f95200a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95380772d0 tx=0x7f952c00b410 comp rx=0 tx=0).stop 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 msgr2=0x7f9538199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 --2- 172.21.15.37:0/1167301661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95381024f0 0x7f9538199170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f95280098e0 tx=0x7f95280057c0 comp rx=0 tx=0).stop 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f9536d9d700 1 -- 172.21.15.37:0/1167301661 reap_dead start 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 shutdown_connections 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 >> 172.21.15.37:0/1167301661 conn(0x7f9538076c60 msgr2=0x7f95381014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 shutdown_connections 2024-04-25T21:06:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:18.896+0000 7f953cc9e700 1 -- 172.21.15.37:0/1167301661 wait complete. 2024-04-25T21:06:19.226 INFO:teuthology.orchestra.run.smithi037.stdout:309237645348 2024-04-25T21:06:19.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 -- 172.21.15.37:0/2988018534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 msgr2=0x7fa320108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 --2- 172.21.15.37:0/2988018534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa320108e30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa308009990 tx=0x7fa308009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 -- 172.21.15.37:0/2988018534 shutdown_connections 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 --2- 172.21.15.37:0/2988018534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 0x7fa32010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 --2- 172.21.15.37:0/2988018534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa320108e30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 --2- 172.21.15.37:0/2988018534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa320102a90 0x7fa320102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 -- 172.21.15.37:0/2988018534 >> 172.21.15.37:0/2988018534 conn(0x7fa320079ba0 msgr2=0x7fa320079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 -- 172.21.15.37:0/2988018534 shutdown_connections 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.925+0000 7fa326e18700 1 -- 172.21.15.37:0/2988018534 wait complete. 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 Processor -- start 2024-04-25T21:06:19.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 -- start start 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa320102a90 0x7fa320074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 0x7fa320075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa3200757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112cb0 con 0x7fa320108a50 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112b30 con 0x7fa320103450 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa326e18700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa320112e30 con 0x7fa320102a90 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa3253b5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa3200757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa3253b5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa3200757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59078/0 (socket says 172.21.15.37:59078) 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa3253b5700 1 -- 172.21.15.37:0/2282685510 learned_addr learned my addr 172.21.15.37:0/2282685510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.926+0000 7fa324bb4700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa320102a90 0x7fa320074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa31ffff700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 0x7fa320075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 -- 172.21.15.37:0/2282685510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa320102a90 msgr2=0x7fa320074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa320102a90 0x7fa320074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 -- 172.21.15.37:0/2282685510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 msgr2=0x7fa320075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 0x7fa320075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3201a7790 con 0x7fa320108a50 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa31ffff700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa320103450 0x7fa320075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa3253b5700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa3200757c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa318007ee0 tx=0x7fa31800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31800dbd0 con 0x7fa320108a50 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31800dd70 con 0x7fa320108a50 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3201a7a80 con 0x7fa320108a50 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318005990 con 0x7fa320108a50 2024-04-25T21:06:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.927+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3201a7fc0 con 0x7fa320108a50 2024-04-25T21:06:19.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.928+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa320063110 con 0x7fa320108a50 2024-04-25T21:06:19.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.929+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa31801a020 con 0x7fa320108a50 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31dffb700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa30c0a3d50 0x7fa30c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa324bb4700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa30c0a3d50 0x7fa30c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fa3180d0410 con 0x7fa320108a50 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31dffb700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] conn(0x7fa30c0ad890 0x7fa30c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa30c0b0400 con 0x7fa30c0ad890 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa3180d0860 con 0x7fa320108a50 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa31ffff700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] conn(0x7fa30c0ad890 0x7fa30c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.930+0000 7fa324bb4700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa30c0a3d50 0x7fa30c0a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa30800b5c0 tx=0x7fa30801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.931+0000 7fa31ffff700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] conn(0x7fa30c0ad890 0x7fa30c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:19.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.931+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== osd.0 v2:172.21.15.37:6802/1094372261 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c0b0400 con 0x7fa30c0ad890 2024-04-25T21:06:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.948+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 --> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa320063590 con 0x7fa30c0ad890 2024-04-25T21:06:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.948+0000 7fa31dffb700 1 -- 172.21.15.37:0/2282685510 <== osd.0 v2:172.21.15.37:6802/1094372261 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa320063590 con 0x7fa30c0ad890 2024-04-25T21:06:19.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.948+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] conn(0x7fa30c0ad890 msgr2=0x7fa30c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.948+0000 7fa326e18700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] conn(0x7fa30c0ad890 0x7fa30c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa30c0a3d50 msgr2=0x7fa30c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fa30c0a3d50 0x7fa30c0a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa30800b5c0 tx=0x7fa30801a040 comp rx=0 tx=0).stop 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 msgr2=0x7fa3200757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 --2- 172.21.15.37:0/2282685510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa320108a50 0x7fa3200757c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa318007ee0 tx=0x7fa31800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa3253b5700 1 -- 172.21.15.37:0/2282685510 reap_dead start 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 shutdown_connections 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 >> 172.21.15.37:0/2282685510 conn(0x7fa320079ba0 msgr2=0x7fa320101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 shutdown_connections 2024-04-25T21:06:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:19.949+0000 7fa326e18700 1 -- 172.21.15.37:0/2282685510 wait complete. 2024-04-25T21:06:19.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:19 smithi037 ceph-mon[29195]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:19 smithi098 ceph-mon[32569]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:20.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:19 smithi177 ceph-mon[33043]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:20.587 INFO:teuthology.orchestra.run.smithi037.stdout:141733920837 2024-04-25T21:06:20.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2955937716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8107990 msgr2=0x7fd5c8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2955937716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8107990 0x7fd5c8107d70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009990 tx=0x7fd5b8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2955937716 shutdown_connections 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2955937716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8102300 0x7fd5c810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2955937716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8107990 0x7fd5c8107d70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2955937716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c8101940 0x7fd5c8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2955937716 >> 172.21.15.37:0/2955937716 conn(0x7fd5c80788f0 msgr2=0x7fd5c8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2955937716 shutdown_connections 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.134+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2955937716 wait complete. 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 Processor -- start 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 -- start start 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c8101940 0x7fd5c81038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 0x7fd5c8103e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8107990 0x7fd5c8104360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8111b60 con 0x7fd5c8102300 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c81119e0 con 0x7fd5c8107990 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.135+0000 7fd5cdfec700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8111ce0 con 0x7fd5c8101940 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 0x7fd5c8103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 0x7fd5c8103e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59096/0 (socket says 172.21.15.37:59096) 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 -- 172.21.15.37:0/2306037461 learned_addr learned my addr 172.21.15.37:0/2306037461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c77fe700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c8101940 0x7fd5c81038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c7fff700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8107990 0x7fd5c8104360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 -- 172.21.15.37:0/2306037461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c8101940 msgr2=0x7fd5c81038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5c8101940 0x7fd5c81038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 -- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8107990 msgr2=0x7fd5c8104360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8107990 0x7fd5c8104360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c81a76d0 con 0x7fd5c8102300 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c7fff700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd5c8107990 0x7fd5c8104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c6ffd700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 0x7fd5c8103e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00b530 tx=0x7fd5bc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc015020 con 0x7fd5c8102300 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5bc011ce0 con 0x7fd5c8102300 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c81a79c0 con 0x7fd5c8102300 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc010e40 con 0x7fd5c8102300 2024-04-25T21:06:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.136+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81a7f00 con 0x7fd5c8102300 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.137+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5c806a970 con 0x7fd5c8102300 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.139+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd5bc018020 con 0x7fd5c8102300 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.139+0000 7fd5c4ff9700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd5b00a3cd0 0x7fd5b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.139+0000 7fd5c77fe700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd5b00a3cd0 0x7fd5b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.139+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fd5bc0d0390 con 0x7fd5c8102300 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c4ff9700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] conn(0x7fd5b00ad810 0x7fd5b00afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5b00b0380 con 0x7fd5b00ad810 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fd5bc0d07e0 con 0x7fd5c8102300 2024-04-25T21:06:21.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c77fe700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd5b00a3cd0 0x7fd5b00a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800b5c0 tx=0x7fd5b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c7fff700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] conn(0x7fd5b00ad810 0x7fd5b00afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.140+0000 7fd5c7fff700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] conn(0x7fd5b00ad810 0x7fd5b00afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.141+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== osd.1 v2:172.21.15.98:6800/1749616660 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5b00b0380 con 0x7fd5b00ad810 2024-04-25T21:06:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.158+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 --> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5c8103060 con 0x7fd5b00ad810 2024-04-25T21:06:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.158+0000 7fd5c4ff9700 1 -- 172.21.15.37:0/2306037461 <== osd.1 v2:172.21.15.98:6800/1749616660 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5c8103060 con 0x7fd5b00ad810 2024-04-25T21:06:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] conn(0x7fd5b00ad810 msgr2=0x7fd5b00afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] conn(0x7fd5b00ad810 0x7fd5b00afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd5b00a3cd0 msgr2=0x7fd5b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd5b00a3cd0 0x7fd5b00a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800b5c0 tx=0x7fd5b8005960 comp rx=0 tx=0).stop 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 msgr2=0x7fd5c8103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 --2- 172.21.15.37:0/2306037461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8102300 0x7fd5c8103e20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00b530 tx=0x7fd5bc00b840 comp rx=0 tx=0).stop 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5c7fff700 1 -- 172.21.15.37:0/2306037461 reap_dead start 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 shutdown_connections 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 >> 172.21.15.37:0/2306037461 conn(0x7fd5c80788f0 msgr2=0x7fd5c8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 shutdown_connections 2024-04-25T21:06:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.159+0000 7fd5cdfec700 1 -- 172.21.15.37:0/2306037461 wait complete. 2024-04-25T21:06:22.095 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:21 smithi037 ceph-mon[29195]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:22.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:21 smithi098 ceph-mon[32569]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:22.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:21 smithi177 ceph-mon[33043]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:22.889 INFO:teuthology.orchestra.run.smithi037.stdout:34359738456 2024-04-25T21:06:22.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T21:06:23.531 INFO:teuthology.orchestra.run.smithi037.stdout:77309411407 2024-04-25T21:06:23.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 -- 172.21.15.37:0/988565629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c10f4f0 msgr2=0x7ff30c1118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 --2- 172.21.15.37:0/988565629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c10f4f0 0x7ff30c1118e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff30400b0a0 tx=0x7ff30400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 -- 172.21.15.37:0/988565629 shutdown_connections 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 --2- 172.21.15.37:0/988565629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c10f4f0 0x7ff30c1118e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 --2- 172.21.15.37:0/988565629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c06a750 0x7ff30c06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 --2- 172.21.15.37:0/988565629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c06b070 0x7ff30c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 -- 172.21.15.37:0/988565629 >> 172.21.15.37:0/988565629 conn(0x7ff30c0788f0 msgr2=0x7ff30c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 -- 172.21.15.37:0/988565629 shutdown_connections 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.895+0000 7ff3137ab700 1 -- 172.21.15.37:0/988565629 wait complete. 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 Processor -- start 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 -- start start 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c06a750 0x7ff30c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 0x7ff30c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c10f4f0 0x7ff30c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c114e70 con 0x7ff30c06b070 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c114cf0 con 0x7ff30c10f4f0 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff3137ab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c114ff0 con 0x7ff30c06a750 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff310d46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 0x7ff30c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff310d46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 0x7ff30c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59104/0 (socket says 172.21.15.37:59104) 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff310d46700 1 -- 172.21.15.37:0/70880229 learned_addr learned my addr 172.21.15.37:0/70880229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:23.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff311547700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c06a750 0x7ff30c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.896+0000 7ff311d48700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c10f4f0 0x7ff30c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 -- 172.21.15.37:0/70880229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c06a750 msgr2=0x7ff30c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff30c06a750 0x7ff30c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 -- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c10f4f0 msgr2=0x7ff30c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c10f4f0 0x7ff30c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c1a0380 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff311d48700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff30c10f4f0 0x7ff30c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff310d46700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 0x7ff30c19b900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00d980 tx=0x7ff2fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc017070 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1a0670 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2fc005020 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc009b50 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.897+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1a7ef0 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.898+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff30c063190 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.899+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff2fc0051d0 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff3027fc700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff2f80a3cd0 0x7ff2f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff2fc0d0a40 con 0x7ff30c06b070 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff311547700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff2f80a3cd0 0x7ff2f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff3027fc700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] conn(0x7ff2f80ad810 0x7ff2f80afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:23.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff2f80b0380 con 0x7ff2f80ad810 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc0d0e90 con 0x7ff30c06b070 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff311547700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff2f80a3cd0 0x7ff2f80a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff308009a10 tx=0x7ff308009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.900+0000 7ff311d48700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] conn(0x7ff2f80ad810 0x7ff2f80afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.901+0000 7ff311d48700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] conn(0x7ff2f80ad810 0x7ff2f80afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.902+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== osd.7 v2:172.21.15.98:6816/3844783138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff2f80b0380 con 0x7ff2f80ad810 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.919+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 --> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff30c063610 con 0x7ff2f80ad810 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3027fc700 1 -- 172.21.15.37:0/70880229 <== osd.7 v2:172.21.15.98:6816/3844783138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff30c063610 con 0x7ff2f80ad810 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] conn(0x7ff2f80ad810 msgr2=0x7ff2f80afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] conn(0x7ff2f80ad810 0x7ff2f80afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff2f80a3cd0 msgr2=0x7ff2f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff2f80a3cd0 0x7ff2f80a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff308009a10 tx=0x7ff308009190 comp rx=0 tx=0).stop 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 msgr2=0x7ff30c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:23.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 --2- 172.21.15.37:0/70880229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff30c06b070 0x7ff30c19b900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00d980 tx=0x7ff2fc00dc90 comp rx=0 tx=0).stop 2024-04-25T21:06:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff311d48700 1 -- 172.21.15.37:0/70880229 reap_dead start 2024-04-25T21:06:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 shutdown_connections 2024-04-25T21:06:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 >> 172.21.15.37:0/70880229 conn(0x7ff30c0788f0 msgr2=0x7ff30c1109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 shutdown_connections 2024-04-25T21:06:23.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:21.920+0000 7ff3137ab700 1 -- 172.21.15.37:0/70880229 wait complete. 2024-04-25T21:06:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:23 smithi098 ceph-mon[32569]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:24.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:23 smithi037 ceph-mon[29195]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:24.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:23 smithi177 ceph-mon[33043]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:24.567 INFO:teuthology.orchestra.run.smithi037.stdout:343597383712 2024-04-25T21:06:24.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T21:06:25.493 INFO:teuthology.orchestra.run.smithi037.stdout:287762808874 2024-04-25T21:06:25.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T21:06:26.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:25 smithi037 ceph-mon[29195]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 -- 172.21.15.37:0/1965116856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0074c70 msgr2=0x7ff0a0075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/1965116856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0074c70 0x7ff0a0075050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff090009990 tx=0x7ff090009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 -- 172.21.15.37:0/1965116856 shutdown_connections 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/1965116856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 0x7ff0a0111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/1965116856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0074c70 0x7ff0a0075050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/1965116856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0075590 0x7ff0a010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 -- 172.21.15.37:0/1965116856 >> 172.21.15.37:0/1965116856 conn(0x7ff0a0079ba0 msgr2=0x7ff0a0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 -- 172.21.15.37:0/1965116856 shutdown_connections 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.102+0000 7ff0a4a83700 1 -- 172.21.15.37:0/1965116856 wait complete. 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 Processor -- start 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 -- start start 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0074c70 0x7ff0a019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 0x7ff0a019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 0x7ff0a019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0114e20 con 0x7ff0a0075590 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0114ca0 con 0x7ff0a0074c70 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff0a4a83700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a0114fa0 con 0x7ff0a010f440 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff09dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 0x7ff0a019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff09dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 0x7ff0a019b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59116/0 (socket says 172.21.15.37:59116) 2024-04-25T21:06:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff09dd9b700 1 -- 172.21.15.37:0/3425534462 learned_addr learned my addr 172.21.15.37:0/3425534462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff09ed9d700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 0x7ff0a019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.103+0000 7ff09e59c700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0074c70 0x7ff0a019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 -- 172.21.15.37:0/3425534462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 msgr2=0x7ff0a019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 0x7ff0a019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 -- 172.21.15.37:0/3425534462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0074c70 msgr2=0x7ff0a019b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0074c70 0x7ff0a019b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a01a02e0 con 0x7ff0a0075590 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09ed9d700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0a010f440 0x7ff0a019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09e59c700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0a0074c70 0x7ff0a019b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff09dd9b700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 0x7ff0a019b810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d950 tx=0x7ff09400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094016070 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a01a05d0 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01a7e50 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff094010460 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.104+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09400f620 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.105+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff080001030 con 0x7ff0a0075590 2024-04-25T21:06:26.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.107+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff094019070 con 0x7ff0a0075590 2024-04-25T21:06:26.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff08f7fe700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff0880a3cd0 0x7ff0880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ff0940d04a0 con 0x7ff0a0075590 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff09e59c700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff0880a3cd0 0x7ff0880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff08f7fe700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] conn(0x7ff0880ad810 0x7ff0880afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff094014e80 con 0x7ff0880ad810 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.108+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ff0940d08f0 con 0x7ff0a0075590 2024-04-25T21:06:26.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.109+0000 7ff09ed9d700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] conn(0x7ff0880ad810 0x7ff0880afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.109+0000 7ff09e59c700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff0880a3cd0 0x7ff0880a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff090006010 tx=0x7ff09001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.109+0000 7ff09ed9d700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] conn(0x7ff0880ad810 0x7ff0880afcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.110+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== osd.6 v2:172.21.15.37:6818/3655908250 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff094014e80 con 0x7ff0880ad810 2024-04-25T21:06:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.127+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 --> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff080002e60 con 0x7ff0880ad810 2024-04-25T21:06:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.127+0000 7ff08f7fe700 1 -- 172.21.15.37:0/3425534462 <== osd.6 v2:172.21.15.37:6818/3655908250 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff080002e60 con 0x7ff0880ad810 2024-04-25T21:06:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] conn(0x7ff0880ad810 msgr2=0x7ff0880afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] conn(0x7ff0880ad810 0x7ff0880afcb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff0880a3cd0 msgr2=0x7ff0880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ff0880a3cd0 0x7ff0880a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff090006010 tx=0x7ff09001a040 comp rx=0 tx=0).stop 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 msgr2=0x7ff0a019b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 --2- 172.21.15.37:0/3425534462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0a0075590 0x7ff0a019b810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d950 tx=0x7ff09400dd10 comp rx=0 tx=0).stop 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff09ed9d700 1 -- 172.21.15.37:0/3425534462 reap_dead start 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 shutdown_connections 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 >> 172.21.15.37:0/3425534462 conn(0x7ff0a0079ba0 msgr2=0x7ff0a00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 shutdown_connections 2024-04-25T21:06:26.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.128+0000 7ff0a4a83700 1 -- 172.21.15.37:0/3425534462 wait complete. 2024-04-25T21:06:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:25 smithi098 ceph-mon[32569]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:26.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:25 smithi177 ceph-mon[33043]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 -- 172.21.15.37:0/1349673255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c108a70 msgr2=0x7f8c2c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 --2- 172.21.15.37:0/1349673255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c108a70 0x7f8c2c108e50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009990 tx=0x7f8c28009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 -- 172.21.15.37:0/1349673255 shutdown_connections 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 --2- 172.21.15.37:0/1349673255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c103460 0x7f8c2c10f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 --2- 172.21.15.37:0/1349673255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c108a70 0x7f8c2c108e50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 --2- 172.21.15.37:0/1349673255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 -- 172.21.15.37:0/1349673255 >> 172.21.15.37:0/1349673255 conn(0x7f8c2c079b10 msgr2=0x7f8c2c079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 -- 172.21.15.37:0/1349673255 shutdown_connections 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 -- 172.21.15.37:0/1349673255 wait complete. 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.425+0000 7f8c333fa700 1 Processor -- start 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 -- start start 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c2c103460 0x7f8c2c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c108a70 0x7f8c2c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c112c10 con 0x7f8c2c102ac0 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c112a90 con 0x7f8c2c108a70 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c333fa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c112d90 con 0x7f8c2c103460 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59122/0 (socket says 172.21.15.37:59122) 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 -- 172.21.15.37:0/2461533858 learned_addr learned my addr 172.21.15.37:0/2461533858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31997700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c108a70 0x7f8c2c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c30995700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c2c103460 0x7f8c2c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 -- 172.21.15.37:0/2461533858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c2c103460 msgr2=0x7f8c2c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c2c103460 0x7f8c2c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 -- 172.21.15.37:0/2461533858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c108a70 msgr2=0x7f8c2c19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8c2c108a70 0x7f8c2c19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.426+0000 7f8c31196700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c19e110 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c31196700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c199100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000c00 tx=0x7f8c280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28027070 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c19e3a0 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28003f20 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28022ca0 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c1a5c80 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.427+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c2c0631d0 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.429+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8c2802a070 con 0x7f8c2c102ac0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.429+0000 7f8c227fc700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f8c180a3cd0 0x7f8c180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.429+0000 7f8c30995700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f8c180a3cd0 0x7f8c180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f8c280d1a70 con 0x7f8c2c102ac0 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c227fc700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] conn(0x7f8c180ad810 0x7f8c180afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c280095d0 con 0x7f8c180ad810 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8c280d1ec0 con 0x7f8c2c102ac0 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c31997700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] conn(0x7f8c180ad810 0x7f8c180afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c30995700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f8c180a3cd0 0x7f8c180a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c07a180 tx=0x7f8c1c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.430+0000 7f8c31997700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] conn(0x7f8c180ad810 0x7f8c180afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.431+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== osd.3 v2:172.21.15.37:6810/3382087416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c280095d0 con 0x7f8c180ad810 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.447+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 --> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8c2c063650 con 0x7f8c180ad810 2024-04-25T21:06:26.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c227fc700 1 -- 172.21.15.37:0/2461533858 <== osd.3 v2:172.21.15.37:6810/3382087416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8c2c063650 con 0x7f8c180ad810 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] conn(0x7f8c180ad810 msgr2=0x7f8c180afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] conn(0x7f8c180ad810 0x7f8c180afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f8c180a3cd0 msgr2=0x7f8c180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f8c180a3cd0 0x7f8c180a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c07a180 tx=0x7f8c1c00a300 comp rx=0 tx=0).stop 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 msgr2=0x7f8c2c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 --2- 172.21.15.37:0/2461533858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c2c102ac0 0x7f8c2c199100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c28000c00 tx=0x7f8c280047b0 comp rx=0 tx=0).stop 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c31997700 1 -- 172.21.15.37:0/2461533858 reap_dead start 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 shutdown_connections 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 >> 172.21.15.37:0/2461533858 conn(0x7f8c2c079b10 msgr2=0x7f8c2c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 shutdown_connections 2024-04-25T21:06:26.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:26.448+0000 7f8c333fa700 1 -- 172.21.15.37:0/2461533858 wait complete. 2024-04-25T21:06:27.447 INFO:teuthology.orchestra.run.smithi037.stdout:55834574933 2024-04-25T21:06:27.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T21:06:27.771 INFO:teuthology.orchestra.run.smithi037.stdout:219043332150 2024-04-25T21:06:27.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T21:06:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:27 smithi037 ceph-mon[29195]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T21:06:28.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:27 smithi177 ceph-mon[33043]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T21:06:28.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:27 smithi098 ceph-mon[32569]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-25T21:06:28.468 INFO:teuthology.orchestra.run.smithi037.stdout:197568495675 2024-04-25T21:06:28.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T21:06:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.663+0000 7f8538f6d700 1 -- 172.21.15.37:0/2466589632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534101940 msgr2=0x7f8534101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.663+0000 7f8538f6d700 1 --2- 172.21.15.37:0/2466589632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534101940 0x7f8534101dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8528009990 tx=0x7f8528009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 -- 172.21.15.37:0/2466589632 shutdown_connections 2024-04-25T21:06:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 --2- 172.21.15.37:0/2466589632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534102300 0x7f853410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 --2- 172.21.15.37:0/2466589632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8534107990 0x7f8534107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 --2- 172.21.15.37:0/2466589632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534101940 0x7f8534101dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 -- 172.21.15.37:0/2466589632 >> 172.21.15.37:0/2466589632 conn(0x7f85340788f0 msgr2=0x7f8534078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 -- 172.21.15.37:0/2466589632 shutdown_connections 2024-04-25T21:06:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.664+0000 7f8538f6d700 1 -- 172.21.15.37:0/2466589632 wait complete. 2024-04-25T21:06:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 Processor -- start 2024-04-25T21:06:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 -- start start 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8534101940 0x7f8534197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534102300 0x7f8534198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 0x7f853419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8534111ae0 con 0x7f8534107990 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8534111960 con 0x7f8534102300 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8538f6d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8534111c60 con 0x7f8534101940 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 0x7f853419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8532d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 0x7f853419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47624/0 (socket says 172.21.15.37:47624) 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8532d9d700 1 -- 172.21.15.37:0/4023387263 learned_addr learned my addr 172.21.15.37:0/4023387263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f8531d9b700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534102300 0x7f8534198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.665+0000 7f853259c700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8534101940 0x7f8534197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 -- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8534101940 msgr2=0x7f8534197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8534101940 0x7f8534197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 -- 172.21.15.37:0/4023387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534102300 msgr2=0x7f8534198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8534102300 0x7f8534198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853419d000 con 0x7f8534107990 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8532d9d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 0x7f853419c8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f852c007ee0 tx=0x7f852c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c004bb0 con 0x7f8534107990 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c004d50 con 0x7f8534107990 2024-04-25T21:06:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.666+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853419d2f0 con 0x7f8534107990 2024-04-25T21:06:28.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.667+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c005920 con 0x7f8534107990 2024-04-25T21:06:28.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.667+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85341a4b70 con 0x7f8534107990 2024-04-25T21:06:28.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.667+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8534063190 con 0x7f8534107990 2024-04-25T21:06:28.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.668+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f852c01a020 con 0x7f8534107990 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f85237fe700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f851c0a3d50 0x7f851c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f852c0cfb30 con 0x7f8534107990 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f853259c700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f851c0a3d50 0x7f851c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f85237fe700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] conn(0x7f851c0ad890 0x7f851c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f852c016ec0 con 0x7f851c0ad890 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.669+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f852c0d42c0 con 0x7f8534107990 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.670+0000 7f8531d9b700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] conn(0x7f851c0ad890 0x7f851c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.670+0000 7f853259c700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f851c0a3d50 0x7f851c0a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f852400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.670+0000 7f8531d9b700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] conn(0x7f851c0ad890 0x7f851c0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.671+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== osd.5 v2:172.21.15.177:6808/4266716104 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f852c016ec0 con 0x7f851c0ad890 2024-04-25T21:06:28.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.690+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 --> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8534063610 con 0x7f851c0ad890 2024-04-25T21:06:28.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.691+0000 7f85237fe700 1 -- 172.21.15.37:0/4023387263 <== osd.5 v2:172.21.15.177:6808/4266716104 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8534063610 con 0x7f851c0ad890 2024-04-25T21:06:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] conn(0x7f851c0ad890 msgr2=0x7f851c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] conn(0x7f851c0ad890 0x7f851c0afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f851c0a3d50 msgr2=0x7f851c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f851c0a3d50 0x7f851c0a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8524005950 tx=0x7f852400a300 comp rx=0 tx=0).stop 2024-04-25T21:06:28.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 msgr2=0x7f853419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 --2- 172.21.15.37:0/4023387263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8534107990 0x7f853419c8c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f852c007ee0 tx=0x7f852c00d3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8532d9d700 1 -- 172.21.15.37:0/4023387263 reap_dead start 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 shutdown_connections 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 >> 172.21.15.37:0/4023387263 conn(0x7f85340788f0 msgr2=0x7f85340ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.692+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 shutdown_connections 2024-04-25T21:06:28.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:28.693+0000 7f8538f6d700 1 -- 172.21.15.37:0/4023387263 wait complete. 2024-04-25T21:06:28.917 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:29.466 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:29.482 INFO:teuthology.orchestra.run.smithi037.stdout:163208757313 2024-04-25T21:06:29.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T21:06:29.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:29 smithi037 ceph-mon[29195]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:30.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:29 smithi177 ceph-mon[33043]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:29 smithi098 ceph-mon[32569]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 -- 172.21.15.37:0/1117391133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 msgr2=0x7f36f406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 --2- 172.21.15.37:0/1117391133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f406b670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009990 tx=0x7f36e4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 -- 172.21.15.37:0/1117391133 shutdown_connections 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 --2- 172.21.15.37:0/1117391133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36f406bbb0 0x7f36f410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 --2- 172.21.15.37:0/1117391133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f4108a60 0x7f36f4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 --2- 172.21.15.37:0/1117391133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f406b670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 -- 172.21.15.37:0/1117391133 >> 172.21.15.37:0/1117391133 conn(0x7f36f40789e0 msgr2=0x7f36f4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 -- 172.21.15.37:0/1117391133 shutdown_connections 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.348+0000 7f36fa381700 1 -- 172.21.15.37:0/1117391133 wait complete. 2024-04-25T21:06:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.349+0000 7f36fa381700 1 Processor -- start 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.349+0000 7f36fa381700 1 -- start start 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f41019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f406bbb0 0x7f36f4101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36f4108a60 0x7f36f4102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4112ca0 con 0x7f36f406b1f0 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4112b20 con 0x7f36f4108a60 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36f4112e20 con 0x7f36f406bbb0 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f41019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f41019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47634/0 (socket says 172.21.15.37:47634) 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 -- 172.21.15.37:0/1051974325 learned_addr learned my addr 172.21.15.37:0/1051974325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f37fe700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f406bbb0 0x7f36f4101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f891e700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36f4108a60 0x7f36f4102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:31.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 -- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f406bbb0 msgr2=0x7f36f4101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36f406bbb0 0x7f36f4101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 -- 172.21.15.37:0/1051974325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36f4108a60 msgr2=0x7f36f4102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36f4108a60 0x7f36f4102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f41a76f0 con 0x7f36f406b1f0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f3fff700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f41019b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b530 tx=0x7f36e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8015020 con 0x7f36f406b1f0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f41a79e0 con 0x7f36f406b1f0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36e8011ce0 con 0x7f36f406b1f0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.350+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8010e40 con 0x7f36f406b1f0 2024-04-25T21:06:31.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.351+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f41a7ea0 con 0x7f36f406b1f0 2024-04-25T21:06:31.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.351+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36f40630d0 con 0x7f36f406b1f0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.353+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f36e8018020 con 0x7f36f406b1f0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.353+0000 7f36f17fa700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f36dc0a3cd0 0x7f36dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.353+0000 7f36f37fe700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f36dc0a3cd0 0x7f36dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.353+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f36e80d0430 con 0x7f36f406b1f0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f17fa700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] conn(0x7f36dc0ad810 0x7f36dc0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36dc0b0380 con 0x7f36dc0ad810 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f36e80d0880 con 0x7f36f406b1f0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f37fe700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f36dc0a3cd0 0x7f36dc0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36e400b5c0 tx=0x7f36e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f891e700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] conn(0x7f36dc0ad810 0x7f36dc0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:31.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.354+0000 7f36f891e700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] conn(0x7f36dc0ad810 0x7f36dc0afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:31.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.355+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== osd.8 v2:172.21.15.177:6816/396846600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36dc0b0380 con 0x7f36dc0ad810 2024-04-25T21:06:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.373+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 --> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36f4063550 con 0x7f36dc0ad810 2024-04-25T21:06:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.374+0000 7f36f17fa700 1 -- 172.21.15.37:0/1051974325 <== osd.8 v2:172.21.15.177:6816/396846600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36f4063550 con 0x7f36dc0ad810 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] conn(0x7f36dc0ad810 msgr2=0x7f36dc0afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] conn(0x7f36dc0ad810 0x7f36dc0afcb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f36dc0a3cd0 msgr2=0x7f36dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f36dc0a3cd0 0x7f36dc0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f36e400b5c0 tx=0x7f36e4005960 comp rx=0 tx=0).stop 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 msgr2=0x7f36f41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 --2- 172.21.15.37:0/1051974325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36f406b1f0 0x7f36f41019b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f36e800b530 tx=0x7f36e800b840 comp rx=0 tx=0).stop 2024-04-25T21:06:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36f891e700 1 -- 172.21.15.37:0/1051974325 reap_dead start 2024-04-25T21:06:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 shutdown_connections 2024-04-25T21:06:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 >> 172.21.15.37:0/1051974325 conn(0x7f36f40789e0 msgr2=0x7f36f40ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 shutdown_connections 2024-04-25T21:06:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:31.375+0000 7f36fa381700 1 -- 172.21.15.37:0/1051974325 wait complete. 2024-04-25T21:06:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:31 smithi037 ceph-mon[29195]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:32.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:31 smithi177 ceph-mon[33043]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:32.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:31 smithi098 ceph-mon[32569]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:32.704 INFO:teuthology.orchestra.run.smithi037.stdout:107374182475 2024-04-25T21:06:32.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T21:06:32.716 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:32.717 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:32.718 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:33.961 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:33.962 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:33 smithi037 ceph-mon[29195]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:34.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:33 smithi177 ceph-mon[33043]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 -- 172.21.15.37:0/3661304058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57081022f0 msgr2=0x7f570810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 --2- 172.21.15.37:0/3661304058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57081022f0 0x7f570810e7e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009990 tx=0x7f56f8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 -- 172.21.15.37:0/3661304058 shutdown_connections 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 --2- 172.21.15.37:0/3661304058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57081022f0 0x7f570810e7e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 --2- 172.21.15.37:0/3661304058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5708107980 0x7f5708107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 --2- 172.21.15.37:0/3661304058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 0x7f5708101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 -- 172.21.15.37:0/3661304058 >> 172.21.15.37:0/3661304058 conn(0x7f57080788b0 msgr2=0x7f5708078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 -- 172.21.15.37:0/3661304058 shutdown_connections 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.413+0000 7f570ef87700 1 -- 172.21.15.37:0/3661304058 wait complete. 2024-04-25T21:06:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 Processor -- start 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 -- start start 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 0x7f570819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57081022f0 0x7f570819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5708111b50 con 0x7f5708107980 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f57081119d0 con 0x7f5708101930 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570ef87700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5708111cd0 con 0x7f57081022f0 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570d524700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570d524700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47664/0 (socket says 172.21.15.37:47664) 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.414+0000 7f570d524700 1 -- 172.21.15.37:0/284019011 learned_addr learned my addr 172.21.15.37:0/284019011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f5707fff700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57081022f0 0x7f570819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570cd23700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 0x7f570819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 -- 172.21.15.37:0/284019011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57081022f0 msgr2=0x7f570819b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57081022f0 0x7f570819b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 -- 172.21.15.37:0/284019011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 msgr2=0x7f570819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 0x7f570819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 -- 172.21.15.37:0/284019011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57081a02f0 con 0x7f5708107980 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570d524700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56f80047b0 tx=0x7f56f8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8026070 con 0x7f5708107980 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56f8004c90 con 0x7f5708107980 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57081a0580 con 0x7f5708107980 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.415+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8003cd0 con 0x7f5708107980 2024-04-25T21:06:34.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.416+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57081a7e60 con 0x7f5708107980 2024-04-25T21:06:34.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:33 smithi098 ceph-mon[32569]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.417+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5708069730 con 0x7f5708107980 2024-04-25T21:06:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.418+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56f8029070 con 0x7f5708107980 2024-04-25T21:06:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.419+0000 7f5705ffb700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 0x7f56f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.419+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f56f80d0d90 con 0x7f5708107980 2024-04-25T21:06:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.420+0000 7f570cd23700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 0x7f56f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.420+0000 7f570cd23700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 0x7f56f00a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f570000a400 tx=0x7f570000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:34.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.424+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56f805dff0 con 0x7f5708107980 2024-04-25T21:06:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.634+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f57081a8030 con 0x7f5708107980 2024-04-25T21:06:34.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.634+0000 7f5705ffb700 1 -- 172.21.15.37:0/284019011 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f56f8013070 con 0x7f5708107980 2024-04-25T21:06:34.634 INFO:teuthology.orchestra.run.smithi037.stdout:287762808877 2024-04-25T21:06:34.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 msgr2=0x7f56f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:34.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 0x7f56f00a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f570000a400 tx=0x7f570000d040 comp rx=0 tx=0).stop 2024-04-25T21:06:34.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 msgr2=0x7f570819fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f56f80047b0 tx=0x7f56f8005e80 comp rx=0 tx=0).stop 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 shutdown_connections 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5708107980 0x7f570819fbb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f56f00a3d50 0x7f56f00a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5708101930 0x7f570819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 --2- 172.21.15.37:0/284019011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57081022f0 0x7f570819b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 >> 172.21.15.37:0/284019011 conn(0x7f57080788b0 msgr2=0x7f570806af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 shutdown_connections 2024-04-25T21:06:34.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:34.637+0000 7f570ef87700 1 -- 172.21.15.37:0/284019011 wait complete. 2024-04-25T21:06:35.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:34 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/284019011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T21:06:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:34 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/284019011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T21:06:35.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:34 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/284019011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T21:06:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:35 smithi037 ceph-mon[29195]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:36.335 INFO:teuthology.orchestra.run.smithi037.stdout:253403070514 2024-04-25T21:06:36.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T21:06:36.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:35 smithi177 ceph-mon[33043]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:36.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:35 smithi098 ceph-mon[32569]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:38.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:37 smithi037 ceph-mon[29195]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:37 smithi177 ceph-mon[33043]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:38.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:37 smithi098 ceph-mon[32569]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:38.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 -- 172.21.15.37:0/2413910823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 msgr2=0x7efc6410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:38.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 --2- 172.21.15.37:0/2413910823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6410e7f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b0a0 tx=0x7efc5c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 -- 172.21.15.37:0/2413910823 shutdown_connections 2024-04-25T21:06:38.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 --2- 172.21.15.37:0/2413910823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6410e7f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 --2- 172.21.15.37:0/2413910823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 0x7efc64107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 --2- 172.21.15.37:0/2413910823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc64101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 -- 172.21.15.37:0/2413910823 >> 172.21.15.37:0/2413910823 conn(0x7efc640788f0 msgr2=0x7efc64078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 -- 172.21.15.37:0/2413910823 shutdown_connections 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.982+0000 7efc6a863700 1 -- 172.21.15.37:0/2413910823 wait complete. 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 Processor -- start 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 -- start start 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 0x7efc641970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc64111b50 con 0x7efc64102300 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc641119d0 con 0x7efc64101940 2024-04-25T21:06:38.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.983+0000 7efc6a863700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efc64111cd0 con 0x7efc64107990 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:55038/0 (socket says 172.21.15.37:55038) 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 -- 172.21.15.37:0/1303392872 learned_addr learned my addr 172.21.15.37:0/1303392872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc68e00700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 0x7efc641970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc637fe700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 -- 172.21.15.37:0/1303392872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 msgr2=0x7efc641970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 0x7efc641970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 -- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 msgr2=0x7efc6419f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6419f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 -- 172.21.15.37:0/1303392872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc641979c0 con 0x7efc64101940 2024-04-25T21:06:38.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.984+0000 7efc63fff700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d980 tx=0x7efc5400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.985+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54016070 con 0x7efc64101940 2024-04-25T21:06:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.985+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc54004830 con 0x7efc64101940 2024-04-25T21:06:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.985+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc64197cb0 con 0x7efc64101940 2024-04-25T21:06:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.985+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc54005540 con 0x7efc64101940 2024-04-25T21:06:38.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.985+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641b4a80 con 0x7efc64101940 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.986+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc64103060 con 0x7efc64101940 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.987+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc54004220 con 0x7efc64101940 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.987+0000 7efc617fa700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 0x7efc4c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.988+0000 7efc637fe700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 0x7efc4c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.988+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7efc540d02d0 con 0x7efc64101940 2024-04-25T21:06:38.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.988+0000 7efc637fe700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 0x7efc4c0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc64101760 tx=0x7efc58009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:38.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:38.993+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc5401f020 con 0x7efc64101940 2024-04-25T21:06:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.202+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7efc641b4ce0 con 0x7efc64101940 2024-04-25T21:06:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.203+0000 7efc617fa700 1 -- 172.21.15.37:0/1303392872 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc54022350 con 0x7efc64101940 2024-04-25T21:06:39.202 INFO:teuthology.orchestra.run.smithi037.stdout:197568495678 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 msgr2=0x7efc4c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 0x7efc4c0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efc64101760 tx=0x7efc58009190 comp rx=0 tx=0).stop 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 msgr2=0x7efc6419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efc5400d980 tx=0x7efc5400dc90 comp rx=0 tx=0).stop 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 shutdown_connections 2024-04-25T21:06:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efc64107990 0x7efc641970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efc4c0a3d50 0x7efc4c0a6210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc64101940 0x7efc6419ed20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 --2- 172.21.15.37:0/1303392872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc64102300 0x7efc6419f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 >> 172.21.15.37:0/1303392872 conn(0x7efc640788f0 msgr2=0x7efc6406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 shutdown_connections 2024-04-25T21:06:39.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:39.205+0000 7efc6a863700 1 -- 172.21.15.37:0/1303392872 wait complete. 2024-04-25T21:06:40.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:39 smithi037 ceph-mon[29195]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T21:06:40.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:39 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1303392872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T21:06:40.192 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:39 smithi177 ceph-mon[33043]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T21:06:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:39 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1303392872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T21:06:40.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:39 smithi098 ceph-mon[32569]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-04-25T21:06:40.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:39 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1303392872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T21:06:41.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808874 got 287762808877 for osd.9 2024-04-25T21:06:41.697 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:42.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:41 smithi037 ceph-mon[29195]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:42.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:41 smithi177 ceph-mon[33043]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:42.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:41 smithi098 ceph-mon[32569]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:44.167 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:43 smithi037 ceph-mon[29195]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:44.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:43 smithi177 ceph-mon[33043]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:44.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:43 smithi098 ceph-mon[32569]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:44.521 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495675 got 197568495678 for osd.6 2024-04-25T21:06:44.521 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 -- 172.21.15.37:0/2197612083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc103450 msgr2=0x7f58dc10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 --2- 172.21.15.37:0/2197612083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc103450 0x7f58dc10f8b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58d400b0a0 tx=0x7f58d400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 -- 172.21.15.37:0/2197612083 shutdown_connections 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 --2- 172.21.15.37:0/2197612083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc103450 0x7f58dc10f8b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 --2- 172.21.15.37:0/2197612083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc108a50 0x7f58dc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 --2- 172.21.15.37:0/2197612083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc102a90 0x7f58dc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 -- 172.21.15.37:0/2197612083 >> 172.21.15.37:0/2197612083 conn(0x7f58dc079ba0 msgr2=0x7f58dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 -- 172.21.15.37:0/2197612083 shutdown_connections 2024-04-25T21:06:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.713+0000 7f58e11a0700 1 -- 172.21.15.37:0/2197612083 wait complete. 2024-04-25T21:06:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.714+0000 7f58e11a0700 1 Processor -- start 2024-04-25T21:06:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.714+0000 7f58e11a0700 1 -- start start 2024-04-25T21:06:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc102a90 0x7f58dc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc103450 0x7f58dc199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc112c20 con 0x7f58dc108a50 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc112aa0 con 0x7f58dc102a90 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58e11a0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc112da0 con 0x7f58dc103450 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49024/0 (socket says 172.21.15.37:49024) 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 -- 172.21.15.37:0/3652511640 learned_addr learned my addr 172.21.15.37:0/3652511640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58dad9d700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc102a90 0x7f58dc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58da59c700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc103450 0x7f58dc199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 -- 172.21.15.37:0/3652511640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc103450 msgr2=0x7f58dc199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc103450 0x7f58dc199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 -- 172.21.15.37:0/3652511640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc102a90 msgr2=0x7f58dc199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc102a90 0x7f58dc199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 -- 172.21.15.37:0/3652511640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc19e0d0 con 0x7f58dc108a50 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58db59e700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58d40062a0 tx=0x7f58d4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.715+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4021070 con 0x7f58dc108a50 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.716+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc19e360 con 0x7f58dc108a50 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.716+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4007c80 con 0x7f58dc108a50 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.716+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d401d450 con 0x7f58dc108a50 2024-04-25T21:06:44.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.716+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1a5c40 con 0x7f58dc108a50 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.717+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc069730 con 0x7f58dc108a50 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.718+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58d4004020 con 0x7f58dc108a50 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.718+0000 7f58d3fff700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 0x7f58c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.718+0000 7f58dad9d700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 0x7f58c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.718+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f58d400f020 con 0x7f58dc108a50 2024-04-25T21:06:44.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.719+0000 7f58dad9d700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 0x7f58c80a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005950 tx=0x7f58cc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.723+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58d402a020 con 0x7f58dc108a50 2024-04-25T21:06:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.934+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f58dc02df80 con 0x7f58dc108a50 2024-04-25T21:06:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.935+0000 7f58d3fff700 1 -- 172.21.15.37:0/3652511640 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58d409e580 con 0x7f58dc108a50 2024-04-25T21:06:44.935 INFO:teuthology.orchestra.run.smithi037.stdout:219043332154 2024-04-25T21:06:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 msgr2=0x7f58c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 0x7f58c80a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005950 tx=0x7f58cc00a300 comp rx=0 tx=0).stop 2024-04-25T21:06:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 msgr2=0x7f58dc19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:44.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58d40062a0 tx=0x7f58d4004e20 comp rx=0 tx=0).stop 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 shutdown_connections 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58dc108a50 0x7f58dc19d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f58c80a3cd0 0x7f58c80a6190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58dc102a90 0x7f58dc199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 --2- 172.21.15.37:0/3652511640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58dc103450 0x7f58dc199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 >> 172.21.15.37:0/3652511640 conn(0x7f58dc079ba0 msgr2=0x7f58dc101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 shutdown_connections 2024-04-25T21:06:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:44.937+0000 7f58e11a0700 1 -- 172.21.15.37:0/3652511640 wait complete. 2024-04-25T21:06:45.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:44 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3652511640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T21:06:45.318 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:44 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3652511640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T21:06:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:44 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3652511640' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T21:06:46.080 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:45 smithi037 ceph-mon[29195]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:46.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:45 smithi177 ceph-mon[33043]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:46.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:45 smithi098 ceph-mon[32569]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 -- 172.21.15.37:0/476808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374102ac0 msgr2=0x7f3374102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 --2- 172.21.15.37:0/476808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374102ac0 0x7f3374102f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3368009990 tx=0x7f3368009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 -- 172.21.15.37:0/476808085 shutdown_connections 2024-04-25T21:06:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 --2- 172.21.15.37:0/476808085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374103480 0x7f337410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 --2- 172.21.15.37:0/476808085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 0x7f3374108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 --2- 172.21.15.37:0/476808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374102ac0 0x7f3374102f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 -- 172.21.15.37:0/476808085 >> 172.21.15.37:0/476808085 conn(0x7f3374079bf0 msgr2=0x7f337407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 -- 172.21.15.37:0/476808085 shutdown_connections 2024-04-25T21:06:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 -- 172.21.15.37:0/476808085 wait complete. 2024-04-25T21:06:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.613+0000 7f3379245700 1 Processor -- start 2024-04-25T21:06:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 -- start start 2024-04-25T21:06:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374102ac0 0x7f337419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 0x7f337419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112cf0 con 0x7f3374103480 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112b70 con 0x7f3374102ac0 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3379245700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112e70 con 0x7f3374108ab0 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f337259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f337259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49032/0 (socket says 172.21.15.37:49032) 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f337259c700 1 -- 172.21.15.37:0/2438614943 learned_addr learned my addr 172.21.15.37:0/2438614943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f3372d9d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374102ac0 0x7f337419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.614+0000 7f337359e700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 0x7f337419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 -- 172.21.15.37:0/2438614943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 msgr2=0x7f337419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 0x7f337419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 -- 172.21.15.37:0/2438614943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374102ac0 msgr2=0x7f337419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374102ac0 0x7f337419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 -- 172.21.15.37:0/2438614943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33741a03c0 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f337259c700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3368000c00 tx=0x7f33680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.615+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368027070 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.616+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3368003f20 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.616+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33741a0650 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.616+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368022ca0 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.616+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a7f30 con 0x7f3374103480 2024-04-25T21:06:46.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.618+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33741041f0 con 0x7f3374103480 2024-04-25T21:06:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.618+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f336802a070 con 0x7f3374103480 2024-04-25T21:06:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.619+0000 7f335bfff700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 0x7f335c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.619+0000 7f3372d9d700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 0x7f335c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.619+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f33680d12b0 con 0x7f3374103480 2024-04-25T21:06:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.619+0000 7f3372d9d700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 0x7f335c0a62e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f33640045b0 tx=0x7f336400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.624+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f336805eff0 con 0x7f3374103480 2024-04-25T21:06:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.834+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f337419c710 con 0x7f3374103480 2024-04-25T21:06:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.835+0000 7f335bfff700 1 -- 172.21.15.37:0/2438614943 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3368013070 con 0x7f3374103480 2024-04-25T21:06:46.835 INFO:teuthology.orchestra.run.smithi037.stdout:34359738461 2024-04-25T21:06:46.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 msgr2=0x7f335c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:46.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 0x7f335c0a62e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f33640045b0 tx=0x7f336400b3c0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 msgr2=0x7f337419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:46.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3368000c00 tx=0x7f33680047b0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 shutdown_connections 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3374108ab0 0x7f337419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f335c0a3e20 0x7f335c0a62e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3374102ac0 0x7f337419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 --2- 172.21.15.37:0/2438614943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3374103480 0x7f337419b8f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 >> 172.21.15.37:0/2438614943 conn(0x7f3374079bf0 msgr2=0x7f3374100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 shutdown_connections 2024-04-25T21:06:46.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:46.837+0000 7f3379245700 1 -- 172.21.15.37:0/2438614943 wait complete. 2024-04-25T21:06:47.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332150 got 219043332154 for osd.7 2024-04-25T21:06:47.080 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:47.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:46 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2438614943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T21:06:47.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:46 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2438614943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T21:06:47.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:46 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2438614943' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 -- 172.21.15.37:0/2307720871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124108a70 msgr2=0x7fe124108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 --2- 172.21.15.37:0/2307720871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124108a70 0x7fe124108e50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe114009990 tx=0x7fe114009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 -- 172.21.15.37:0/2307720871 shutdown_connections 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 --2- 172.21.15.37:0/2307720871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124103480 0x7fe12410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 --2- 172.21.15.37:0/2307720871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124108a70 0x7fe124108e50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 --2- 172.21.15.37:0/2307720871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124102ac0 0x7fe124102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 -- 172.21.15.37:0/2307720871 >> 172.21.15.37:0/2307720871 conn(0x7fe124079bb0 msgr2=0x7fe124079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 -- 172.21.15.37:0/2307720871 shutdown_connections 2024-04-25T21:06:47.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 -- 172.21.15.37:0/2307720871 wait complete. 2024-04-25T21:06:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.843+0000 7fe12887d700 1 Processor -- start 2024-04-25T21:06:47.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 -- start start 2024-04-25T21:06:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:47.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124103480 0x7fe1241995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 0x7fe12419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112c00 con 0x7fe124102ac0 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112a80 con 0x7fe124108a70 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12887d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe124112d80 con 0x7fe124103480 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49046/0 (socket says 172.21.15.37:49046) 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 -- 172.21.15.37:0/2054066325 learned_addr learned my addr 172.21.15.37:0/2054066325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe121d9b700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124103480 0x7fe1241995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe122d9d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 0x7fe12419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 -- 172.21.15.37:0/2054066325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124103480 msgr2=0x7fe1241995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124103480 0x7fe1241995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 -- 172.21.15.37:0/2054066325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 msgr2=0x7fe12419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:47.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 0x7fe12419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 -- 172.21.15.37:0/2054066325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12419e0a0 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe122d9d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 0x7fe12419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.844+0000 7fe12259c700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe1140057c0 tx=0x7fe1140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.845+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114027070 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.845+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe114003f20 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.845+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114022c30 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.845+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12419e330 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.845+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1241a5c10 con 0x7fe124102ac0 2024-04-25T21:06:47.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.846+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1241041a0 con 0x7fe124102ac0 2024-04-25T21:06:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.847+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe11402a070 con 0x7fe124102ac0 2024-04-25T21:06:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.848+0000 7fe1137fe700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 0x7fe10c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.848+0000 7fe121d9b700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 0x7fe10c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.848+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe1140d1c70 con 0x7fe124102ac0 2024-04-25T21:06:47.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.848+0000 7fe121d9b700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 0x7fe10c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe12407a120 tx=0x7fe11800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:47.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:47.851+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe114031020 con 0x7fe124102ac0 2024-04-25T21:06:48.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:47 smithi037 ceph-mon[29195]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 -- 172.21.15.37:0/148318570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8102300 msgr2=0x7fced810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 --2- 172.21.15.37:0/148318570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8102300 0x7fced810e7f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fced000b0a0 tx=0x7fced000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 -- 172.21.15.37:0/148318570 shutdown_connections 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 --2- 172.21.15.37:0/148318570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8102300 0x7fced810e7f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 --2- 172.21.15.37:0/148318570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 --2- 172.21.15.37:0/148318570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8101940 0x7fced8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 -- 172.21.15.37:0/148318570 >> 172.21.15.37:0/148318570 conn(0x7fced80788f0 msgr2=0x7fced8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 -- 172.21.15.37:0/148318570 shutdown_connections 2024-04-25T21:06:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.043+0000 7fcedea90700 1 -- 172.21.15.37:0/148318570 wait complete. 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 Processor -- start 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 -- start start 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8102300 0x7fced819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fced8111b20 con 0x7fced8101940 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fced81119a0 con 0x7fced8102300 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedea90700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fced8111ca0 con 0x7fced8107990 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedc82c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedc82c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49056/0 (socket says 172.21.15.37:49056) 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedc82c700 1 -- 172.21.15.37:0/1977539735 learned_addr learned my addr 172.21.15.37:0/1977539735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fced7fff700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8102300 0x7fced819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.044+0000 7fcedd02d700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 -- 172.21.15.37:0/1977539735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 msgr2=0x7fced819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 -- 172.21.15.37:0/1977539735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8102300 msgr2=0x7fced819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8102300 0x7fced819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 -- 172.21.15.37:0/1977539735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced81a0330 con 0x7fced8101940 2024-04-25T21:06:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedd02d700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedc82c700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c920 tx=0x7fcecc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007b00 con 0x7fced8101940 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced81a0620 con 0x7fced8101940 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcecc004ce0 con 0x7fced8101940 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc005500 con 0x7fced8101940 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.045+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced81a7ea0 con 0x7fced8101940 2024-04-25T21:06:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.046+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced8069730 con 0x7fced8101940 2024-04-25T21:06:48.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.048+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcecc007ca0 con 0x7fced8101940 2024-04-25T21:06:48.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.049+0000 7fced5ffb700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 0x7fcec80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.049+0000 7fced7fff700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 0x7fcec80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.049+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fcecc0cffc0 con 0x7fced8101940 2024-04-25T21:06:48.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.049+0000 7fced7fff700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 0x7fcec80a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fced810eba0 tx=0x7fcec400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.058+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcecc014030 con 0x7fced8101940 2024-04-25T21:06:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.075+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe12402df80 con 0x7fe124102ac0 2024-04-25T21:06:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.075+0000 7fe1137fe700 1 -- 172.21.15.37:0/2054066325 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe114013070 con 0x7fe124102ac0 2024-04-25T21:06:48.076 INFO:teuthology.orchestra.run.smithi037.stdout:141733920842 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 msgr2=0x7fe10c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 0x7fe10c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe12407a120 tx=0x7fe11800b410 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 msgr2=0x7fe124199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe1140057c0 tx=0x7fe1140057f0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 shutdown_connections 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe10c0a3cd0 0x7fe10c0a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe124108a70 0x7fe12419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe124102ac0 0x7fe124199090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 --2- 172.21.15.37:0/2054066325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe124103480 0x7fe1241995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.078+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 >> 172.21.15.37:0/2054066325 conn(0x7fe124079bb0 msgr2=0x7fe1241016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.079+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 shutdown_connections 2024-04-25T21:06:48.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.079+0000 7fe12887d700 1 -- 172.21.15.37:0/2054066325 wait complete. 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 -- 172.21.15.37:0/3674690733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8102300 msgr2=0x7f04d810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 --2- 172.21.15.37:0/3674690733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8102300 0x7f04d810e7f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009990 tx=0x7f04d0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 -- 172.21.15.37:0/3674690733 shutdown_connections 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 --2- 172.21.15.37:0/3674690733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8102300 0x7f04d810e7f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 --2- 172.21.15.37:0/3674690733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8107990 0x7f04d8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 --2- 172.21.15.37:0/3674690733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 0x7f04d8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 -- 172.21.15.37:0/3674690733 >> 172.21.15.37:0/3674690733 conn(0x7f04d80788f0 msgr2=0x7f04d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 -- 172.21.15.37:0/3674690733 shutdown_connections 2024-04-25T21:06:48.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.256+0000 7f04dddc8700 1 -- 172.21.15.37:0/3674690733 wait complete. 2024-04-25T21:06:48.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 Processor -- start 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 -- start start 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 0x7f04d81034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8102300 0x7f04d8103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111b00 con 0x7f04d8107990 2024-04-25T21:06:48.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111980 con 0x7f04d8101940 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04dddc8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f04d8111c80 con 0x7f04d8102300 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04d7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49066/0 (socket says 172.21.15.37:49066) 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04d7fff700 1 -- 172.21.15.37:0/1496996484 learned_addr learned my addr 172.21.15.37:0/1496996484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.257+0000 7f04d6ffd700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8102300 0x7f04d8103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d77fe700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 0x7f04d81034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 -- 172.21.15.37:0/1496996484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8102300 msgr2=0x7f04d8103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8102300 0x7f04d8103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 -- 172.21.15.37:0/1496996484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 msgr2=0x7f04d81034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 0x7f04d81034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 -- 172.21.15.37:0/1496996484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d81a7610 con 0x7f04d8107990 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d7fff700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c010 tx=0x7f04d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:48.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0027070 con 0x7f04d8107990 2024-04-25T21:06:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0022470 con 0x7f04d8107990 2024-04-25T21:06:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d81a78a0 con 0x7f04d8107990 2024-04-25T21:06:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0017650 con 0x7f04d8107990 2024-04-25T21:06:48.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.258+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81a7cf0 con 0x7f04d8107990 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.259+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d8069730 con 0x7f04d8107990 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.260+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04d002a070 con 0x7f04d8107990 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.261+0000 7f04d4ff9700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 0x7f04c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.261+0000 7f04d77fe700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 0x7f04c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.261+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f04d00d1290 con 0x7f04d8107990 2024-04-25T21:06:48.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.262+0000 7f04d77fe700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 0x7f04c00a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04c800f540 tx=0x7f04c800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:48.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.266+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04d0013070 con 0x7f04d8107990 2024-04-25T21:06:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.288+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fced81a8170 con 0x7fced8101940 2024-04-25T21:06:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.288+0000 7fced5ffb700 1 -- 172.21.15.37:0/1977539735 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcecc09c8f0 con 0x7fced8101940 2024-04-25T21:06:48.289 INFO:teuthology.orchestra.run.smithi037.stdout:163208757317 2024-04-25T21:06:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 msgr2=0x7fcec80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 0x7fcec80a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fced810eba0 tx=0x7fcec400b410 comp rx=0 tx=0).stop 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 msgr2=0x7fced819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcecc00c920 tx=0x7fcecc00cc30 comp rx=0 tx=0).stop 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 shutdown_connections 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fced8107990 0x7fced819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fced8101940 0x7fced819b320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fcec80a3d50 0x7fcec80a6210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 --2- 172.21.15.37:0/1977539735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fced8102300 0x7fced819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.291+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 >> 172.21.15.37:0/1977539735 conn(0x7fced80788f0 msgr2=0x7fced806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:48.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.292+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 shutdown_connections 2024-04-25T21:06:48.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.292+0000 7fcedea90700 1 -- 172.21.15.37:0/1977539735 wait complete. 2024-04-25T21:06:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:47 smithi177 ceph-mon[33043]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:48.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:47 smithi098 ceph-mon[32569]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:48.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.492+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f04d802e1e0 con 0x7f04d8107990 2024-04-25T21:06:48.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.492+0000 7f04d4ff9700 1 -- 172.21.15.37:0/1496996484 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f04d009dbb0 con 0x7f04d8107990 2024-04-25T21:06:48.493 INFO:teuthology.orchestra.run.smithi037.stdout:55834574938 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 msgr2=0x7f04c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 0x7f04c00a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f04c800f540 tx=0x7f04c800b040 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 msgr2=0x7f04d8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c010 tx=0x7f04d00047b0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 shutdown_connections 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04d8107990 0x7f04d8103f70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f04c00a3cd0 0x7f04c00a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f04d8101940 0x7f04d81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 --2- 172.21.15.37:0/1496996484 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04d8102300 0x7f04d8103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 >> 172.21.15.37:0/1496996484 conn(0x7f04d80788f0 msgr2=0x7f04d806aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 shutdown_connections 2024-04-25T21:06:48.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:48.494+0000 7f04dddc8700 1 -- 172.21.15.37:0/1496996484 wait complete. 2024-04-25T21:06:49.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2054066325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T21:06:49.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1977539735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T21:06:49.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1496996484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T21:06:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2054066325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T21:06:49.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1977539735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T21:06:49.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1496996484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T21:06:49.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2054066325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T21:06:49.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1977539735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T21:06:49.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1496996484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T21:06:49.485 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920837 got 141733920842 for osd.4 2024-04-25T21:06:49.485 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:50.092 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:49 smithi037 ceph-mon[29195]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:50.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:49 smithi177 ceph-mon[33043]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:50.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:49 smithi098 ceph-mon[32569]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:50.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738461 for osd.0 2024-04-25T21:06:50.444 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:50.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 -- 172.21.15.37:0/1789656673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c101030 msgr2=0x7fe95c101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:50.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 --2- 172.21.15.37:0/1789656673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c101030 0x7fe95c101490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009990 tx=0x7fe94c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 -- 172.21.15.37:0/1789656673 shutdown_connections 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 --2- 172.21.15.37:0/1789656673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c1019d0 0x7fe95c10f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 --2- 172.21.15.37:0/1789656673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c108ad0 0x7fe95c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 --2- 172.21.15.37:0/1789656673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c101030 0x7fe95c101490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 -- 172.21.15.37:0/1789656673 >> 172.21.15.37:0/1789656673 conn(0x7fe95c077fb0 msgr2=0x7fe95c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 -- 172.21.15.37:0/1789656673 shutdown_connections 2024-04-25T21:06:50.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.904+0000 7fe963ba6700 1 -- 172.21.15.37:0/1789656673 wait complete. 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 Processor -- start 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 -- start start 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c101030 0x7fe95c102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c108ad0 0x7fe95c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:50.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c112c20 con 0x7fe95c1019d0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c112aa0 con 0x7fe95c101030 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe963ba6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c112da0 con 0x7fe95c108ad0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52236/0 (socket says 172.21.15.37:52236) 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 -- 172.21.15.37:0/3381493675 learned_addr learned my addr 172.21.15.37:0/3381493675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe962143700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c108ad0 0x7fe95c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961942700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c101030 0x7fe95c102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 -- 172.21.15.37:0/3381493675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c108ad0 msgr2=0x7fe95c102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c108ad0 0x7fe95c102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 -- 172.21.15.37:0/3381493675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c101030 msgr2=0x7fe95c102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c101030 0x7fe95c102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.905+0000 7fe961141700 1 -- 172.21.15.37:0/3381493675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c1a76c0 con 0x7fe95c1019d0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe961141700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0057c0 tx=0x7fe94c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:50.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c027070 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c1a7950 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94c003f20 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c022ba0 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.906+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c1a7e30 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.907+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c104350 con 0x7fe95c1019d0 2024-04-25T21:06:50.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.908+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe94c02a070 con 0x7fe95c1019d0 2024-04-25T21:06:50.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.909+0000 7fe952ffd700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 0x7fe9480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:50.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.909+0000 7fe961942700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 0x7fe9480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:50.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.909+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fe94c0d14a0 con 0x7fe95c1019d0 2024-04-25T21:06:50.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.909+0000 7fe961942700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 0x7fe9480a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe9580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:50.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:50.917+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe94c05eff0 con 0x7fe95c1019d0 2024-04-25T21:06:51.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574938 for osd.1 2024-04-25T21:06:51.057 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.156+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe95c02df80 con 0x7fe95c1019d0 2024-04-25T21:06:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.156+0000 7fe952ffd700 1 -- 172.21.15.37:0/3381493675 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe94c013070 con 0x7fe95c1019d0 2024-04-25T21:06:51.157 INFO:teuthology.orchestra.run.smithi037.stdout:343597383719 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.158+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 msgr2=0x7fe9480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.158+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 0x7fe9480a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe958005950 tx=0x7fe9580058e0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.158+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 msgr2=0x7fe95c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.158+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0057c0 tx=0x7fe94c0057f0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 shutdown_connections 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe95c108ad0 0x7fe95c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fe9480a3cd0 0x7fe9480a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe95c101030 0x7fe95c102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 --2- 172.21.15.37:0/3381493675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe95c1019d0 0x7fe95c102750 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 >> 172.21.15.37:0/3381493675 conn(0x7fe95c077fb0 msgr2=0x7fe95c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:51.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 shutdown_connections 2024-04-25T21:06:51.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.159+0000 7fe963ba6700 1 -- 172.21.15.37:0/3381493675 wait complete. 2024-04-25T21:06:51.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 -- 172.21.15.37:0/1369143221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c108a30 msgr2=0x7f461c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 --2- 172.21.15.37:0/1369143221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c108a30 0x7f461c108e10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4618009990 tx=0x7f4618009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 -- 172.21.15.37:0/1369143221 shutdown_connections 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 --2- 172.21.15.37:0/1369143221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c06b0f0 0x7f461c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 --2- 172.21.15.37:0/1369143221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c108a30 0x7f461c108e10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 --2- 172.21.15.37:0/1369143221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06a750 0x7f461c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 -- 172.21.15.37:0/1369143221 >> 172.21.15.37:0/1369143221 conn(0x7f461c0788f0 msgr2=0x7f461c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 -- 172.21.15.37:0/1369143221 shutdown_connections 2024-04-25T21:06:51.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.275+0000 7f4624150700 1 -- 172.21.15.37:0/1369143221 wait complete. 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.276+0000 7f4624150700 1 Processor -- start 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.276+0000 7f4624150700 1 -- start start 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.276+0000 7f4624150700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.276+0000 7f4624150700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06b0f0 0x7f461c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.276+0000 7f4624150700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c108a30 0x7f461c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4624150700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112c30 con 0x7f461c06a750 2024-04-25T21:06:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4624150700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112ab0 con 0x7f461c108a30 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4624150700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f461c112db0 con 0x7f461c06b0f0 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52264/0 (socket says 172.21.15.37:52264) 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 -- 172.21.15.37:0/169216046 learned_addr learned my addr 172.21.15.37:0/169216046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f46226ed700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c108a30 0x7f461c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f46216eb700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06b0f0 0x7f461c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 -- 172.21.15.37:0/169216046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06b0f0 msgr2=0x7f461c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06b0f0 0x7f461c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 -- 172.21.15.37:0/169216046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c108a30 msgr2=0x7f461c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c108a30 0x7f461c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 -- 172.21.15.37:0/169216046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461c19e0e0 con 0x7f461c06a750 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4621eec700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f46180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618027070 con 0x7f461c06a750 2024-04-25T21:06:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461c19e370 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46180040e0 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.277+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800f460 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.278+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461c1a5c50 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.278+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c104170 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.280+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f461802a070 con 0x7f461c06a750 2024-04-25T21:06:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.280+0000 7f4612ffd700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 0x7f46080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:51.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.281+0000 7f46216eb700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 0x7f46080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:51.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.281+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f46180d1540 con 0x7f461c06a750 2024-04-25T21:06:51.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.281+0000 7f46216eb700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 0x7f46080a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.285+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f461805eff0 con 0x7f461c06a750 2024-04-25T21:06:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.496+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f461c19a430 con 0x7f461c06a750 2024-04-25T21:06:51.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.497+0000 7f4612ffd700 1 -- 172.21.15.37:0/169216046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4618013070 con 0x7f461c06a750 2024-04-25T21:06:51.498 INFO:teuthology.orchestra.run.smithi037.stdout:77309411414 2024-04-25T21:06:51.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 msgr2=0x7f46080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 0x7f46080a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f460c006fd0 tx=0x7f460c008040 comp rx=0 tx=0).stop 2024-04-25T21:06:51.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 msgr2=0x7f461c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:51.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4618000c00 tx=0x7f46180047b0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 shutdown_connections 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f46080a3cd0 0x7f46080a6190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f461c108a30 0x7f461c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f461c06a750 0x7f461c199120 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 --2- 172.21.15.37:0/169216046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f461c06b0f0 0x7f461c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 >> 172.21.15.37:0/169216046 conn(0x7f461c0788f0 msgr2=0x7f461c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:51.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 shutdown_connections 2024-04-25T21:06:51.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:51.499+0000 7f4624150700 1 -- 172.21.15.37:0/169216046 wait complete. 2024-04-25T21:06:51.962 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383712 got 343597383719 for osd.11 2024-04-25T21:06:51.962 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:52.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:51 smithi037 ceph-mon[29195]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:51 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3381493675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T21:06:52.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:51 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/169216046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T21:06:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:51 smithi177 ceph-mon[33043]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:52.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:51 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3381493675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T21:06:52.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:51 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/169216046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T21:06:52.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:51 smithi098 ceph-mon[32569]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:06:52.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:51 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3381493675' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T21:06:52.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:51 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/169216046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T21:06:52.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411407 got 77309411414 for osd.2 2024-04-25T21:06:52.623 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:53.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757313 got 163208757317 for osd.5 2024-04-25T21:06:53.434 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:53.474 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:06:53.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:53 smithi037 ceph-mon[29195]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:53 smithi177 ceph-mon[33043]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:54.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:53 smithi098 ceph-mon[32569]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 -- 172.21.15.37:0/640690498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4108a50 msgr2=0x7ffaf4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 --2- 172.21.15.37:0/640690498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4108a50 0x7ffaf4108e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009990 tx=0x7ffae4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 -- 172.21.15.37:0/640690498 shutdown_connections 2024-04-25T21:06:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 --2- 172.21.15.37:0/640690498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 0x7ffaf410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 --2- 172.21.15.37:0/640690498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4108a50 0x7ffaf4108e30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 --2- 172.21.15.37:0/640690498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4102a90 0x7ffaf4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 -- 172.21.15.37:0/640690498 >> 172.21.15.37:0/640690498 conn(0x7ffaf4079ba0 msgr2=0x7ffaf4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 -- 172.21.15.37:0/640690498 shutdown_connections 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.697+0000 7ffaf9964700 1 -- 172.21.15.37:0/640690498 wait complete. 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 Processor -- start 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 -- start start 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 0x7ffaf4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4108a50 0x7ffaf40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112be0 con 0x7ffaf4102a90 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112a60 con 0x7ffaf4108a50 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf9964700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112d60 con 0x7ffaf4103450 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf2ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52290/0 (socket says 172.21.15.37:52290) 2024-04-25T21:06:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.698+0000 7ffaf2ffd700 1 -- 172.21.15.37:0/886869891 learned_addr learned my addr 172.21.15.37:0/886869891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf27fc700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 0x7ffaf4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf37fe700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4108a50 0x7ffaf40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 -- 172.21.15.37:0/886869891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 msgr2=0x7ffaf4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 0x7ffaf4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 -- 172.21.15.37:0/886869891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4108a50 msgr2=0x7ffaf40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4108a50 0x7ffaf40757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 -- 172.21.15.37:0/886869891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf41a7660 con 0x7ffaf4102a90 2024-04-25T21:06:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf2ffd700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000c00 tx=0x7ffae4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4027070 con 0x7ffaf4102a90 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.699+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf41a78f0 con 0x7ffaf4102a90 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.700+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf41a7e20 con 0x7ffaf4102a90 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.700+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4022470 con 0x7ffaf4102a90 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.700+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4017650 con 0x7ffaf4102a90 2024-04-25T21:06:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.701+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf4069730 con 0x7ffaf4102a90 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.702+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffae402a070 con 0x7ffaf4102a90 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.702+0000 7ffaf8962700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 0x7ffadc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.703+0000 7ffaf27fc700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 0x7ffadc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.703+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7ffae40d1370 con 0x7ffaf4102a90 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.703+0000 7ffaf27fc700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 0x7ffadc0a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffaf407a260 tx=0x7ffae8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.707+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffae4013070 con 0x7ffaf4102a90 2024-04-25T21:06:55.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.922+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffaf402df80 con 0x7ffaf4102a90 2024-04-25T21:06:55.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.922+0000 7ffaf8962700 1 -- 172.21.15.37:0/886869891 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffae409dc90 con 0x7ffaf4102a90 2024-04-25T21:06:55.923 INFO:teuthology.orchestra.run.smithi037.stdout:309237645356 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 msgr2=0x7ffadc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 0x7ffadc0a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffaf407a260 tx=0x7ffae8005d50 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 msgr2=0x7ffaf4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000c00 tx=0x7ffae4009580 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 shutdown_connections 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7ffadc0a3d50 0x7ffadc0a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffaf4108a50 0x7ffaf40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaf4102a90 0x7ffaf4074d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 --2- 172.21.15.37:0/886869891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaf4103450 0x7ffaf4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 >> 172.21.15.37:0/886869891 conn(0x7ffaf4079ba0 msgr2=0x7ffaf4101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 shutdown_connections 2024-04-25T21:06:55.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:55.924+0000 7ffaf9964700 1 -- 172.21.15.37:0/886869891 wait complete. 2024-04-25T21:06:56.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:55 smithi037 ceph-mon[29195]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:56.091 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:55 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/886869891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T21:06:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:55 smithi177 ceph-mon[33043]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:56.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:55 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/886869891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T21:06:56.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:55 smithi098 ceph-mon[32569]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:06:56.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:55 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/886869891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T21:06:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- 172.21.15.37:0/3248746714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330074c70 msgr2=0x7fd330075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 --2- 172.21.15.37:0/3248746714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330074c70 0x7fd330075050 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009990 tx=0x7fd32c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- 172.21.15.37:0/3248746714 shutdown_connections 2024-04-25T21:06:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 --2- 172.21.15.37:0/3248746714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330113c40 0x7fd330116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 --2- 172.21.15.37:0/3248746714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330074c70 0x7fd330075050 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 --2- 172.21.15.37:0/3248746714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330075590 0x7fd330113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- 172.21.15.37:0/3248746714 >> 172.21.15.37:0/3248746714 conn(0x7fd330079bb0 msgr2=0x7fd330079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- 172.21.15.37:0/3248746714 shutdown_connections 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- 172.21.15.37:0/3248746714 wait complete. 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 Processor -- start 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.912+0000 7fd337d94700 1 -- start start 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 0x7fd33019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 0x7fd3301a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd330119220 con 0x7fd330075590 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3301190a0 con 0x7fd330113c40 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd337d94700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd3301193a0 con 0x7fd330074c70 2024-04-25T21:06:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52298/0 (socket says 172.21.15.37:52298) 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 -- 172.21.15.37:0/12825305 learned_addr learned my addr 172.21.15.37:0/12825305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd335b30700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 0x7fd33019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd336331700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 0x7fd3301a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 -- 172.21.15.37:0/12825305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 msgr2=0x7fd33019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 0x7fd33019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 -- 172.21.15.37:0/12825305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 msgr2=0x7fd3301a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 0x7fd3301a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 -- 172.21.15.37:0/12825305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3301a4740 con 0x7fd330075590 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd335b30700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 0x7fd33019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd336331700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 0x7fd3301a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.913+0000 7fd33532f700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c920 tx=0x7fd32000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.914+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320007b00 con 0x7fd330075590 2024-04-25T21:06:56.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.914+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3301a4a30 con 0x7fd330075590 2024-04-25T21:06:56.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.914+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd320004ce0 con 0x7fd330075590 2024-04-25T21:06:56.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.914+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3301ac2b0 con 0x7fd330075590 2024-04-25T21:06:56.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.914+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320005650 con 0x7fd330075590 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.915+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd330069730 con 0x7fd330075590 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.916+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd320007ca0 con 0x7fd330075590 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.916+0000 7fd326ffd700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 0x7fd31c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.917+0000 7fd335b30700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 0x7fd31c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.917+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fd3200d04c0 con 0x7fd330075590 2024-04-25T21:06:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.917+0000 7fd335b30700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 0x7fd31c0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd32c006010 tx=0x7fd32c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:56.921+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd320014030 con 0x7fd330075590 2024-04-25T21:06:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.134+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd3301ac5d0 con 0x7fd330075590 2024-04-25T21:06:57.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.134+0000 7fd326ffd700 1 -- 172.21.15.37:0/12825305 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd32009cd50 con 0x7fd330075590 2024-04-25T21:06:57.135 INFO:teuthology.orchestra.run.smithi037.stdout:253403070519 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 msgr2=0x7fd31c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 0x7fd31c0a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd32c006010 tx=0x7fd32c00b540 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 msgr2=0x7fd33019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd32000c920 tx=0x7fd32000cce0 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 shutdown_connections 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fd31c0a3d50 0x7fd31c0a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd330113c40 0x7fd3301a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd330074c70 0x7fd33019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 --2- 172.21.15.37:0/12825305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd330075590 0x7fd33019fc70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:57.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 >> 172.21.15.37:0/12825305 conn(0x7fd330079bb0 msgr2=0x7fd33010f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 shutdown_connections 2024-04-25T21:06:57.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:57.136+0000 7fd337d94700 1 -- 172.21.15.37:0/12825305 wait complete. 2024-04-25T21:06:57.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645348 got 309237645356 for osd.10 2024-04-25T21:06:57.365 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:58.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:57 smithi037 ceph-mon[29195]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:58.100 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:57 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/12825305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T21:06:58.130 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070514 got 253403070519 for osd.8 2024-04-25T21:06:58.131 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- 172.21.15.37:0/2478755554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda24108a60 msgr2=0x7fda24108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 --2- 172.21.15.37:0/2478755554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda24108a60 0x7fda24108e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fda20009990 tx=0x7fda20009ca0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- 172.21.15.37:0/2478755554 shutdown_connections 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 --2- 172.21.15.37:0/2478755554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406bbb0 0x7fda2410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 --2- 172.21.15.37:0/2478755554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda24108a60 0x7fda24108e40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 --2- 172.21.15.37:0/2478755554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda2406b1f0 0x7fda2406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- 172.21.15.37:0/2478755554 >> 172.21.15.37:0/2478755554 conn(0x7fda240789e0 msgr2=0x7fda24078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- 172.21.15.37:0/2478755554 shutdown_connections 2024-04-25T21:06:58.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- 172.21.15.37:0/2478755554 wait complete. 2024-04-25T21:06:58.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 Processor -- start 2024-04-25T21:06:58.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.328+0000 7fda2c322700 1 -- start start 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406b1f0 0x7fda241991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 0x7fda2419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112cf0 con 0x7fda2406bbb0 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112b70 con 0x7fda2406b1f0 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2c322700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fda24112e70 con 0x7fda24108a60 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52310/0 (socket says 172.21.15.37:52310) 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 -- 172.21.15.37:0/4003363567 learned_addr learned my addr 172.21.15.37:0/4003363567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2a0be700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406b1f0 0x7fda241991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2a8bf700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 0x7fda2419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 -- 172.21.15.37:0/4003363567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 msgr2=0x7fda2419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 0x7fda2419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 -- 172.21.15.37:0/4003363567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406b1f0 msgr2=0x7fda241991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406b1f0 0x7fda241991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 -- 172.21.15.37:0/4003363567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2419e1f0 con 0x7fda2406bbb0 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda2a8bf700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 0x7fda2419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda298bd700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b530 tx=0x7fda1400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.329+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14015020 con 0x7fda2406bbb0 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.330+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda14011ce0 con 0x7fda2406bbb0 2024-04-25T21:06:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.330+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2419e4e0 con 0x7fda2406bbb0 2024-04-25T21:06:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.330+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14010e40 con 0x7fda2406bbb0 2024-04-25T21:06:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.330+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda241a5d60 con 0x7fda2406bbb0 2024-04-25T21:06:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.331+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2410b060 con 0x7fda2406bbb0 2024-04-25T21:06:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.332+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fda14018020 con 0x7fda2406bbb0 2024-04-25T21:06:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.332+0000 7fda1b7fe700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 0x7fda100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:06:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.332+0000 7fda2a0be700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 0x7fda100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:06:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.332+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fda140cfcc0 con 0x7fda2406bbb0 2024-04-25T21:06:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.333+0000 7fda2a0be700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 0x7fda100a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b5c0 tx=0x7fda20005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:06:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.337+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda1401f020 con 0x7fda2406bbb0 2024-04-25T21:06:58.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:57 smithi177 ceph-mon[33043]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:58.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:57 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/12825305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T21:06:58.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:57 smithi098 ceph-mon[32569]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:06:58.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:57 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/12825305' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T21:06:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.547+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fda2419a540 con 0x7fda2406bbb0 2024-04-25T21:06:58.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.548+0000 7fda1b7fe700 1 -- 172.21.15.37:0/4003363567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fda1409c660 con 0x7fda2406bbb0 2024-04-25T21:06:58.549 INFO:teuthology.orchestra.run.smithi037.stdout:107374182481 2024-04-25T21:06:58.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 msgr2=0x7fda100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 0x7fda100a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda2000b5c0 tx=0x7fda20005fd0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 msgr2=0x7fda24199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fda1400b530 tx=0x7fda1400b840 comp rx=0 tx=0).stop 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 shutdown_connections 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda24108a60 0x7fda2419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fda100a3cd0 0x7fda100a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda2406b1f0 0x7fda241991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 --2- 172.21.15.37:0/4003363567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda2406bbb0 0x7fda24199720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:06:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 >> 172.21.15.37:0/4003363567 conn(0x7fda240789e0 msgr2=0x7fda240ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:06:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 shutdown_connections 2024-04-25T21:06:58.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:06:58.550+0000 7fda2c322700 1 -- 172.21.15.37:0/4003363567 wait complete. 2024-04-25T21:06:59.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:58 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4003363567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T21:06:59.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:58 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:59.274 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182475 got 107374182481 for osd.3 2024-04-25T21:06:59.275 DEBUG:teuthology.parallel:result is None 2024-04-25T21:06:59.275 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T21:06:59.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:06:59.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:58 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4003363567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T21:06:59.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:58 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:59.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:58 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4003363567' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T21:06:59.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:58 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:00.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:06:59 smithi037 ceph-mon[29195]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:00.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:06:59 smithi177 ceph-mon[33043]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:00.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:06:59 smithi098 ceph-mon[32569]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:01.651 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:01 smithi037 ceph-mon[29195]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:01.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:01 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:01 smithi098 ceph-mon[32569]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:01 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:01 smithi177 ceph-mon[33043]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:01.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:01 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/2887274255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74102a90 msgr2=0x7f9f74102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/2887274255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74102a90 0x7f9f74102f10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009990 tx=0x7f9f60009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/2887274255 shutdown_connections 2024-04-25T21:07:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/2887274255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74103450 0x7f9f7410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/2887274255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74108a50 0x7f9f74108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/2887274255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74102a90 0x7f9f74102f10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/2887274255 >> 172.21.15.37:0/2887274255 conn(0x7f9f74079ba0 msgr2=0x7f9f74079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/2887274255 shutdown_connections 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.930+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/2887274255 wait complete. 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 Processor -- start 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 -- start start 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74102a90 0x7f9f74199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:01.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74108a50 0x7f9f7419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74112be0 con 0x7f9f74103450 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74112a60 con 0x7f9f74102a90 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f7aaa5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9f74112d60 con 0x7f9f74108a50 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33464/0 (socket says 172.21.15.37:33464) 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.931+0000 7f9f73fff700 1 -- 172.21.15.37:0/1346878997 learned_addr learned my addr 172.21.15.37:0/1346878997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f78841700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74102a90 0x7f9f74199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f79042700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74108a50 0x7f9f7419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 -- 172.21.15.37:0/1346878997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74108a50 msgr2=0x7f9f7419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74108a50 0x7f9f7419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 -- 172.21.15.37:0/1346878997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74102a90 msgr2=0x7f9f74199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74102a90 0x7f9f74199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 -- 172.21.15.37:0/1346878997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7419e140 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f73fff700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9f60000c00 tx=0x7f9f6000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60027070 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f60022470 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7419e3d0 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60017650 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.932+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f741a5cb0 con 0x7f9f74103450 2024-04-25T21:07:01.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.933+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f74069730 con 0x7f9f74103450 2024-04-25T21:07:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.935+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f6002a070 con 0x7f9f74103450 2024-04-25T21:07:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.935+0000 7f9f71ffb700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 0x7f9f640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.936+0000 7f9f78841700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 0x7f9f640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.936+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f9f600d1290 con 0x7f9f74103450 2024-04-25T21:07:01.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.936+0000 7f9f78841700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 0x7f9f640a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009be0 tx=0x7f9f68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:01.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:01.941+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f60013070 con 0x7f9f74103450 2024-04-25T21:07:02.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.174+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f7419a490 con 0x7f9f640a3cd0 2024-04-25T21:07:02.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.181+0000 7f9f71ffb700 1 -- 172.21.15.37:0/1346878997 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258917 (secure 0 0 0) 0x7f9f7419a490 con 0x7f9f640a3cd0 2024-04-25T21:07:02.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 msgr2=0x7f9f640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 0x7f9f640a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009be0 tx=0x7f9f68009260 comp rx=0 tx=0).stop 2024-04-25T21:07:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 msgr2=0x7f9f74199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:02.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9f60000c00 tx=0x7f9f6000bfd0 comp rx=0 tx=0).stop 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 shutdown_connections 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f74108a50 0x7f9f7419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9f640a3cd0 0x7f9f640a6190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f74102a90 0x7f9f74199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 --2- 172.21.15.37:0/1346878997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f74103450 0x7f9f74199670 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 >> 172.21.15.37:0/1346878997 conn(0x7f9f74079ba0 msgr2=0x7f9f741017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 shutdown_connections 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:02.185+0000 7f9f7aaa5700 1 -- 172.21.15.37:0/1346878997 wait complete. 2024-04-25T21:07:02.189 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:07:03.027 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-25T21:07:00.226090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1148,"num_read_kb":697,"num_write":427,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361332,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580492,"statfs":{"total":1151940427776,"available":1151570423808,"internally_reserved":0,"allocated":12435456,"data_stored":7724292,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003221"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812561+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:05:35.812561+0000","last_peered":"2024-04-25T21:05:35.812561+0000","last_clean":"2024-04-25T21:05:35.812561+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:05:35.812561+0000","last_undegraded":"2024-04-25T21:05:35.812561+0000","last_fullsized":"2024-04-25T21:05:35.812561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812177+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:05:35.812177+0000","last_peered":"2024-04-25T21:05:35.812177+0000","last_clean":"2024-04-25T21:05:35.812177+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:05:35.812177+0000","last_undegraded":"2024-04-25T21:05:35.812177+0000","last_fullsized":"2024-04-25T21:05:35.812177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811939+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:05:35.811939+0000","last_peered":"2024-04-25T21:05:35.811939+0000","last_clean":"2024-04-25T21:05:35.811939+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:05:35.811939+0000","last_undegraded":"2024-04-25T21:05:35.811939+0000","last_fullsized":"2024-04-25T21:05:35.811939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280134+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:05:36.280134+0000","last_peered":"2024-04-25T21:05:36.280134+0000","last_clean":"2024-04-25T21:05:36.280134+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:05:36.280134+0000","last_undegraded":"2024-04-25T21:05:36.280134+0000","last_fullsized":"2024-04-25T21:05:36.280134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815068+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:05:35.815068+0000","last_peered":"2024-04-25T21:05:35.815068+0000","last_clean":"2024-04-25T21:05:35.815068+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:05:35.815068+0000","last_undegraded":"2024-04-25T21:05:35.815068+0000","last_fullsized":"2024-04-25T21:05:35.815068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054645+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:05:36.054645+0000","last_peered":"2024-04-25T21:05:36.054645+0000","last_clean":"2024-04-25T21:05:36.054645+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:05:36.054645+0000","last_undegraded":"2024-04-25T21:05:36.054645+0000","last_fullsized":"2024-04-25T21:05:36.054645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810131+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:05:35.810131+0000","last_peered":"2024-04-25T21:05:35.810131+0000","last_clean":"2024-04-25T21:05:35.810131+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:05:35.810131+0000","last_undegraded":"2024-04-25T21:05:35.810131+0000","last_fullsized":"2024-04-25T21:05:35.810131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164368+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:05:36.164368+0000","last_peered":"2024-04-25T21:05:36.164368+0000","last_clean":"2024-04-25T21:05:36.164368+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:05:36.164368+0000","last_undegraded":"2024-04-25T21:05:36.164368+0000","last_fullsized":"2024-04-25T21:05:36.164368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812458+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:05:35.812458+0000","last_peered":"2024-04-25T21:05:35.812458+0000","last_clean":"2024-04-25T21:05:35.812458+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:05:35.812458+0000","last_undegraded":"2024-04-25T21:05:35.812458+0000","last_fullsized":"2024-04-25T21:05:35.812458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054978+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:05:36.054978+0000","last_peered":"2024-04-25T21:05:36.054978+0000","last_clean":"2024-04-25T21:05:36.054978+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:05:36.054978+0000","last_undegraded":"2024-04-25T21:05:36.054978+0000","last_fullsized":"2024-04-25T21:05:36.054978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810362+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:05:35.810362+0000","last_peered":"2024-04-25T21:05:35.810362+0000","last_clean":"2024-04-25T21:05:35.810362+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:05:35.810362+0000","last_undegraded":"2024-04-25T21:05:35.810362+0000","last_fullsized":"2024-04-25T21:05:35.810362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809516+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:05:35.809516+0000","last_peered":"2024-04-25T21:05:35.809516+0000","last_clean":"2024-04-25T21:05:35.809516+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:05:35.809516+0000","last_undegraded":"2024-04-25T21:05:35.809516+0000","last_fullsized":"2024-04-25T21:05:35.809516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811795+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:05:35.811795+0000","last_peered":"2024-04-25T21:05:35.811795+0000","last_clean":"2024-04-25T21:05:35.811795+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:05:35.811795+0000","last_undegraded":"2024-04-25T21:05:35.811795+0000","last_fullsized":"2024-04-25T21:05:35.811795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818318+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:05:35.818318+0000","last_peered":"2024-04-25T21:05:35.818318+0000","last_clean":"2024-04-25T21:05:35.818318+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:05:35.818318+0000","last_undegraded":"2024-04-25T21:05:35.818318+0000","last_fullsized":"2024-04-25T21:05:35.818318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811748+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:05:35.811748+0000","last_peered":"2024-04-25T21:05:35.811748+0000","last_clean":"2024-04-25T21:05:35.811748+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:05:35.811748+0000","last_undegraded":"2024-04-25T21:05:35.811748+0000","last_fullsized":"2024-04-25T21:05:35.811748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812733+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:05:35.812733+0000","last_peered":"2024-04-25T21:05:35.812733+0000","last_clean":"2024-04-25T21:05:35.812733+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:05:35.812733+0000","last_undegraded":"2024-04-25T21:05:35.812733+0000","last_fullsized":"2024-04-25T21:05:35.812733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847351+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:05:35.847351+0000","last_peered":"2024-04-25T21:05:35.847351+0000","last_clean":"2024-04-25T21:05:35.847351+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:05:35.847351+0000","last_undegraded":"2024-04-25T21:05:35.847351+0000","last_fullsized":"2024-04-25T21:05:35.847351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813305+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:05:35.813305+0000","last_peered":"2024-04-25T21:05:35.813305+0000","last_clean":"2024-04-25T21:05:35.813305+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:05:35.813305+0000","last_undegraded":"2024-04-25T21:05:35.813305+0000","last_fullsized":"2024-04-25T21:05:35.813305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":139,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:57.386783+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:06:57.386783+0000","last_peered":"2024-04-25T21:06:57.386783+0000","last_clean":"2024-04-25T21:06:57.386783+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:06:57.386783+0000","last_undegraded":"2024-04-25T21:06:57.386783+0000","last_fullsized":"2024-04-25T21:06:57.386783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164650+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:05:36.164650+0000","last_peered":"2024-04-25T21:05:36.164650+0000","last_clean":"2024-04-25T21:05:36.164650+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:05:36.164650+0000","last_undegraded":"2024-04-25T21:05:36.164650+0000","last_fullsized":"2024-04-25T21:05:36.164650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164841+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:05:36.164841+0000","last_peered":"2024-04-25T21:05:36.164841+0000","last_clean":"2024-04-25T21:05:36.164841+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:05:36.164841+0000","last_undegraded":"2024-04-25T21:05:36.164841+0000","last_fullsized":"2024-04-25T21:05:36.164841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812294+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:05:35.812294+0000","last_peered":"2024-04-25T21:05:35.812294+0000","last_clean":"2024-04-25T21:05:35.812294+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:05:35.812294+0000","last_undegraded":"2024-04-25T21:05:35.812294+0000","last_fullsized":"2024-04-25T21:05:35.812294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815926+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:05:35.815926+0000","last_peered":"2024-04-25T21:05:35.815926+0000","last_clean":"2024-04-25T21:05:35.815926+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:05:35.815926+0000","last_undegraded":"2024-04-25T21:05:35.815926+0000","last_fullsized":"2024-04-25T21:05:35.815926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811808+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:05:35.811808+0000","last_peered":"2024-04-25T21:05:35.811808+0000","last_clean":"2024-04-25T21:05:35.811808+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:05:35.811808+0000","last_undegraded":"2024-04-25T21:05:35.811808+0000","last_fullsized":"2024-04-25T21:05:35.811808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.823684+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:05:35.823684+0000","last_peered":"2024-04-25T21:05:35.823684+0000","last_clean":"2024-04-25T21:05:35.823684+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:05:35.823684+0000","last_undegraded":"2024-04-25T21:05:35.823684+0000","last_fullsized":"2024-04-25T21:05:35.823684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810659+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:05:35.810659+0000","last_peered":"2024-04-25T21:05:35.810659+0000","last_clean":"2024-04-25T21:05:35.810659+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:05:35.810659+0000","last_undegraded":"2024-04-25T21:05:35.810659+0000","last_fullsized":"2024-04-25T21:05:35.810659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:58.389544+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:06:58.389544+0000","last_peered":"2024-04-25T21:06:58.389544+0000","last_clean":"2024-04-25T21:06:58.389544+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:06:58.389544+0000","last_undegraded":"2024-04-25T21:06:58.389544+0000","last_fullsized":"2024-04-25T21:06:58.389544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815987+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:05:35.815987+0000","last_peered":"2024-04-25T21:05:35.815987+0000","last_clean":"2024-04-25T21:05:35.815987+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:05:35.815987+0000","last_undegraded":"2024-04-25T21:05:35.815987+0000","last_fullsized":"2024-04-25T21:05:35.815987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054563+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:05:36.054563+0000","last_peered":"2024-04-25T21:05:36.054563+0000","last_clean":"2024-04-25T21:05:36.054563+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:05:36.054563+0000","last_undegraded":"2024-04-25T21:05:36.054563+0000","last_fullsized":"2024-04-25T21:05:36.054563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810324+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:05:35.810324+0000","last_peered":"2024-04-25T21:05:35.810324+0000","last_clean":"2024-04-25T21:05:35.810324+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:05:35.810324+0000","last_undegraded":"2024-04-25T21:05:35.810324+0000","last_fullsized":"2024-04-25T21:05:35.810324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809323+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:05:35.809323+0000","last_peered":"2024-04-25T21:05:35.809323+0000","last_clean":"2024-04-25T21:05:35.809323+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:05:35.809323+0000","last_undegraded":"2024-04-25T21:05:35.809323+0000","last_fullsized":"2024-04-25T21:05:35.809323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164264+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:05:36.164264+0000","last_peered":"2024-04-25T21:05:36.164264+0000","last_clean":"2024-04-25T21:05:36.164264+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:05:36.164264+0000","last_undegraded":"2024-04-25T21:05:36.164264+0000","last_fullsized":"2024-04-25T21:05:36.164264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809991+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:05:35.809991+0000","last_peered":"2024-04-25T21:05:35.809991+0000","last_clean":"2024-04-25T21:05:35.809991+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:05:35.809991+0000","last_undegraded":"2024-04-25T21:05:35.809991+0000","last_fullsized":"2024-04-25T21:05:35.809991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847028+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:05:35.847028+0000","last_peered":"2024-04-25T21:05:35.847028+0000","last_clean":"2024-04-25T21:05:35.847028+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:05:35.847028+0000","last_undegraded":"2024-04-25T21:05:35.847028+0000","last_fullsized":"2024-04-25T21:05:35.847028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":222,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:55.384273+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:06:55.384273+0000","last_peered":"2024-04-25T21:06:55.384273+0000","last_clean":"2024-04-25T21:06:55.384273+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:06:55.384273+0000","last_undegraded":"2024-04-25T21:06:55.384273+0000","last_fullsized":"2024-04-25T21:06:55.384273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812371+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:05:35.812371+0000","last_peered":"2024-04-25T21:05:35.812371+0000","last_clean":"2024-04-25T21:05:35.812371+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:05:35.812371+0000","last_undegraded":"2024-04-25T21:05:35.812371+0000","last_fullsized":"2024-04-25T21:05:35.812371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810524+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:05:35.810524+0000","last_peered":"2024-04-25T21:05:35.810524+0000","last_clean":"2024-04-25T21:05:35.810524+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:05:35.810524+0000","last_undegraded":"2024-04-25T21:05:35.810524+0000","last_fullsized":"2024-04-25T21:05:35.810524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812286+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:05:35.812286+0000","last_peered":"2024-04-25T21:05:35.812286+0000","last_clean":"2024-04-25T21:05:35.812286+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:05:35.812286+0000","last_undegraded":"2024-04-25T21:05:35.812286+0000","last_fullsized":"2024-04-25T21:05:35.812286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810988+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:05:35.810988+0000","last_peered":"2024-04-25T21:05:35.810988+0000","last_clean":"2024-04-25T21:05:35.810988+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:05:35.810988+0000","last_undegraded":"2024-04-25T21:05:35.810988+0000","last_fullsized":"2024-04-25T21:05:35.810988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812018+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:05:35.812018+0000","last_peered":"2024-04-25T21:05:35.812018+0000","last_clean":"2024-04-25T21:05:35.812018+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:05:35.812018+0000","last_undegraded":"2024-04-25T21:05:35.812018+0000","last_fullsized":"2024-04-25T21:05:35.812018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":257,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:55.397153+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:06:55.397153+0000","last_peered":"2024-04-25T21:06:55.397153+0000","last_clean":"2024-04-25T21:06:55.397153+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:06:55.397153+0000","last_undegraded":"2024-04-25T21:06:55.397153+0000","last_fullsized":"2024-04-25T21:06:55.397153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":200,"num_read_kb":158,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815665+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:05:35.815665+0000","last_peered":"2024-04-25T21:05:35.815665+0000","last_clean":"2024-04-25T21:05:35.815665+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:05:35.815665+0000","last_undegraded":"2024-04-25T21:05:35.815665+0000","last_fullsized":"2024-04-25T21:05:35.815665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813128+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:05:35.813128+0000","last_peered":"2024-04-25T21:05:35.813128+0000","last_clean":"2024-04-25T21:05:35.813128+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:05:35.813128+0000","last_undegraded":"2024-04-25T21:05:35.813128+0000","last_fullsized":"2024-04-25T21:05:35.813128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809221+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:05:35.809221+0000","last_peered":"2024-04-25T21:05:35.809221+0000","last_clean":"2024-04-25T21:05:35.809221+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:05:35.809221+0000","last_undegraded":"2024-04-25T21:05:35.809221+0000","last_fullsized":"2024-04-25T21:05:35.809221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810239+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:05:35.810239+0000","last_peered":"2024-04-25T21:05:35.810239+0000","last_clean":"2024-04-25T21:05:35.810239+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:05:35.810239+0000","last_undegraded":"2024-04-25T21:05:35.810239+0000","last_fullsized":"2024-04-25T21:05:35.810239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848492+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:05:35.848492+0000","last_peered":"2024-04-25T21:05:35.848492+0000","last_clean":"2024-04-25T21:05:35.848492+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:05:35.848492+0000","last_undegraded":"2024-04-25T21:05:35.848492+0000","last_fullsized":"2024-04-25T21:05:35.848492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:54.382872+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:06:54.382872+0000","last_peered":"2024-04-25T21:06:54.382872+0000","last_clean":"2024-04-25T21:06:54.382872+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:06:54.382872+0000","last_undegraded":"2024-04-25T21:06:54.382872+0000","last_fullsized":"2024-04-25T21:06:54.382872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809971+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:05:35.809971+0000","last_peered":"2024-04-25T21:05:35.809971+0000","last_clean":"2024-04-25T21:05:35.809971+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:05:35.809971+0000","last_undegraded":"2024-04-25T21:05:35.809971+0000","last_fullsized":"2024-04-25T21:05:35.809971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815899+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:05:35.815899+0000","last_peered":"2024-04-25T21:05:35.815899+0000","last_clean":"2024-04-25T21:05:35.815899+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:05:35.815899+0000","last_undegraded":"2024-04-25T21:05:35.815899+0000","last_fullsized":"2024-04-25T21:05:35.815899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810151+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:05:35.810151+0000","last_peered":"2024-04-25T21:05:35.810151+0000","last_clean":"2024-04-25T21:05:35.810151+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:05:35.810151+0000","last_undegraded":"2024-04-25T21:05:35.810151+0000","last_fullsized":"2024-04-25T21:05:35.810151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815284+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:05:35.815284+0000","last_peered":"2024-04-25T21:05:35.815284+0000","last_clean":"2024-04-25T21:05:35.815284+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:05:35.815284+0000","last_undegraded":"2024-04-25T21:05:35.815284+0000","last_fullsized":"2024-04-25T21:05:35.815284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812340+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:05:35.812340+0000","last_peered":"2024-04-25T21:05:35.812340+0000","last_clean":"2024-04-25T21:05:35.812340+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:05:35.812340+0000","last_undegraded":"2024-04-25T21:05:35.812340+0000","last_fullsized":"2024-04-25T21:05:35.812340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847208+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:05:35.847208+0000","last_peered":"2024-04-25T21:05:35.847208+0000","last_clean":"2024-04-25T21:05:35.847208+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:05:35.847208+0000","last_undegraded":"2024-04-25T21:05:35.847208+0000","last_fullsized":"2024-04-25T21:05:35.847208+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054796+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:05:36.054796+0000","last_peered":"2024-04-25T21:05:36.054796+0000","last_clean":"2024-04-25T21:05:36.054796+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:05:36.054796+0000","last_undegraded":"2024-04-25T21:05:36.054796+0000","last_fullsized":"2024-04-25T21:05:36.054796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":127,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:54.382906+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:06:54.382906+0000","last_peered":"2024-04-25T21:06:54.382906+0000","last_clean":"2024-04-25T21:06:54.382906+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:06:54.382906+0000","last_undegraded":"2024-04-25T21:06:54.382906+0000","last_fullsized":"2024-04-25T21:06:54.382906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054640+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:05:36.054640+0000","last_peered":"2024-04-25T21:05:36.054640+0000","last_clean":"2024-04-25T21:05:36.054640+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:05:36.054640+0000","last_undegraded":"2024-04-25T21:05:36.054640+0000","last_fullsized":"2024-04-25T21:05:36.054640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810247+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:05:35.810247+0000","last_peered":"2024-04-25T21:05:35.810247+0000","last_clean":"2024-04-25T21:05:35.810247+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:05:35.810247+0000","last_undegraded":"2024-04-25T21:05:35.810247+0000","last_fullsized":"2024-04-25T21:05:35.810247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":129,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:56.385204+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:06:56.385204+0000","last_peered":"2024-04-25T21:06:56.385204+0000","last_clean":"2024-04-25T21:06:56.385204+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:06:56.385204+0000","last_undegraded":"2024-04-25T21:06:56.385204+0000","last_fullsized":"2024-04-25T21:06:56.385204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810193+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:05:35.810193+0000","last_peered":"2024-04-25T21:05:35.810193+0000","last_clean":"2024-04-25T21:05:35.810193+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:05:35.810193+0000","last_undegraded":"2024-04-25T21:05:35.810193+0000","last_fullsized":"2024-04-25T21:05:35.810193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809159+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:05:35.809159+0000","last_peered":"2024-04-25T21:05:35.809159+0000","last_clean":"2024-04-25T21:05:35.809159+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:05:35.809159+0000","last_undegraded":"2024-04-25T21:05:35.809159+0000","last_fullsized":"2024-04-25T21:05:35.809159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810226+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:05:35.810226+0000","last_peered":"2024-04-25T21:05:35.810226+0000","last_clean":"2024-04-25T21:05:35.810226+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:05:35.810226+0000","last_undegraded":"2024-04-25T21:05:35.810226+0000","last_fullsized":"2024-04-25T21:05:35.810226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815740+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:05:35.815740+0000","last_peered":"2024-04-25T21:05:35.815740+0000","last_clean":"2024-04-25T21:05:35.815740+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:05:35.815740+0000","last_undegraded":"2024-04-25T21:05:35.815740+0000","last_fullsized":"2024-04-25T21:05:35.815740+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809414+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:05:35.809414+0000","last_peered":"2024-04-25T21:05:35.809414+0000","last_clean":"2024-04-25T21:05:35.809414+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:05:35.809414+0000","last_undegraded":"2024-04-25T21:05:35.809414+0000","last_fullsized":"2024-04-25T21:05:35.809414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:41.673824+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:05:41.673824+0000","last_peered":"2024-04-25T21:05:41.673824+0000","last_clean":"2024-04-25T21:05:41.673824+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:05:41.673824+0000","last_undegraded":"2024-04-25T21:05:41.673824+0000","last_fullsized":"2024-04-25T21:05:41.673824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164820+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:05:36.164820+0000","last_peered":"2024-04-25T21:05:36.164820+0000","last_clean":"2024-04-25T21:05:36.164820+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:05:36.164820+0000","last_undegraded":"2024-04-25T21:05:36.164820+0000","last_fullsized":"2024-04-25T21:05:36.164820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280051+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:05:36.280051+0000","last_peered":"2024-04-25T21:05:36.280051+0000","last_clean":"2024-04-25T21:05:36.280051+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:05:36.280051+0000","last_undegraded":"2024-04-25T21:05:36.280051+0000","last_fullsized":"2024-04-25T21:05:36.280051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812196+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:05:35.812196+0000","last_peered":"2024-04-25T21:05:35.812196+0000","last_clean":"2024-04-25T21:05:35.812196+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:05:35.812196+0000","last_undegraded":"2024-04-25T21:05:35.812196+0000","last_fullsized":"2024-04-25T21:05:35.812196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.820411+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:05:35.820411+0000","last_peered":"2024-04-25T21:05:35.820411+0000","last_clean":"2024-04-25T21:05:35.820411+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:05:35.820411+0000","last_undegraded":"2024-04-25T21:05:35.820411+0000","last_fullsized":"2024-04-25T21:05:35.820411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.816311+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:05:35.816311+0000","last_peered":"2024-04-25T21:05:35.816311+0000","last_clean":"2024-04-25T21:05:35.816311+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:05:35.816311+0000","last_undegraded":"2024-04-25T21:05:35.816311+0000","last_fullsized":"2024-04-25T21:05:35.816311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164359+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:05:36.164359+0000","last_peered":"2024-04-25T21:05:36.164359+0000","last_clean":"2024-04-25T21:05:36.164359+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:05:36.164359+0000","last_undegraded":"2024-04-25T21:05:36.164359+0000","last_fullsized":"2024-04-25T21:05:36.164359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813860+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:05:35.813860+0000","last_peered":"2024-04-25T21:05:35.813860+0000","last_clean":"2024-04-25T21:05:35.813860+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:05:35.813860+0000","last_undegraded":"2024-04-25T21:05:35.813860+0000","last_fullsized":"2024-04-25T21:05:35.813860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812840+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:05:35.812840+0000","last_peered":"2024-04-25T21:05:35.812840+0000","last_clean":"2024-04-25T21:05:35.812840+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:05:35.812840+0000","last_undegraded":"2024-04-25T21:05:35.812840+0000","last_fullsized":"2024-04-25T21:05:35.812840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815193+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:05:35.815193+0000","last_peered":"2024-04-25T21:05:35.815193+0000","last_clean":"2024-04-25T21:05:35.815193+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:05:35.815193+0000","last_undegraded":"2024-04-25T21:05:35.815193+0000","last_fullsized":"2024-04-25T21:05:35.815193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810529+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:05:35.810529+0000","last_peered":"2024-04-25T21:05:35.810529+0000","last_clean":"2024-04-25T21:05:35.810529+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:05:35.810529+0000","last_undegraded":"2024-04-25T21:05:35.810529+0000","last_fullsized":"2024-04-25T21:05:35.810529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:58.402134+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:06:58.402134+0000","last_peered":"2024-04-25T21:06:58.402134+0000","last_clean":"2024-04-25T21:06:58.402134+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:06:58.402134+0000","last_undegraded":"2024-04-25T21:06:58.402134+0000","last_fullsized":"2024-04-25T21:06:58.402134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810542+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:05:35.810542+0000","last_peered":"2024-04-25T21:05:35.810542+0000","last_clean":"2024-04-25T21:05:35.810542+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:05:35.810542+0000","last_undegraded":"2024-04-25T21:05:35.810542+0000","last_fullsized":"2024-04-25T21:05:35.810542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810498+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:05:35.810498+0000","last_peered":"2024-04-25T21:05:35.810498+0000","last_clean":"2024-04-25T21:05:35.810498+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:05:35.810498+0000","last_undegraded":"2024-04-25T21:05:35.810498+0000","last_fullsized":"2024-04-25T21:05:35.810498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810354+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:05:35.810354+0000","last_peered":"2024-04-25T21:05:35.810354+0000","last_clean":"2024-04-25T21:05:35.810354+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:05:35.810354+0000","last_undegraded":"2024-04-25T21:05:35.810354+0000","last_fullsized":"2024-04-25T21:05:35.810354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809534+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:05:35.809534+0000","last_peered":"2024-04-25T21:05:35.809534+0000","last_clean":"2024-04-25T21:05:35.809534+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:05:35.809534+0000","last_undegraded":"2024-04-25T21:05:35.809534+0000","last_fullsized":"2024-04-25T21:05:35.809534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810657+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:05:35.810657+0000","last_peered":"2024-04-25T21:05:35.810657+0000","last_clean":"2024-04-25T21:05:35.810657+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:05:35.810657+0000","last_undegraded":"2024-04-25T21:05:35.810657+0000","last_fullsized":"2024-04-25T21:05:35.810657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848543+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:05:35.848543+0000","last_peered":"2024-04-25T21:05:35.848543+0000","last_clean":"2024-04-25T21:05:35.848543+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:05:35.848543+0000","last_undegraded":"2024-04-25T21:05:35.848543+0000","last_fullsized":"2024-04-25T21:05:35.848543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.814755+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:05:35.814755+0000","last_peered":"2024-04-25T21:05:35.814755+0000","last_clean":"2024-04-25T21:05:35.814755+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:05:35.814755+0000","last_undegraded":"2024-04-25T21:05:35.814755+0000","last_fullsized":"2024-04-25T21:05:35.814755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812713+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:05:35.812713+0000","last_peered":"2024-04-25T21:05:35.812713+0000","last_clean":"2024-04-25T21:05:35.812713+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:05:35.812713+0000","last_undegraded":"2024-04-25T21:05:35.812713+0000","last_fullsized":"2024-04-25T21:05:35.812713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811944+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:05:35.811944+0000","last_peered":"2024-04-25T21:05:35.811944+0000","last_clean":"2024-04-25T21:05:35.811944+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:05:35.811944+0000","last_undegraded":"2024-04-25T21:05:35.811944+0000","last_fullsized":"2024-04-25T21:05:35.811944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809423+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:05:35.809423+0000","last_peered":"2024-04-25T21:05:35.809423+0000","last_clean":"2024-04-25T21:05:35.809423+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:05:35.809423+0000","last_undegraded":"2024-04-25T21:05:35.809423+0000","last_fullsized":"2024-04-25T21:05:35.809423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848642+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:05:35.848642+0000","last_peered":"2024-04-25T21:05:35.848642+0000","last_clean":"2024-04-25T21:05:35.848642+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:05:35.848642+0000","last_undegraded":"2024-04-25T21:05:35.848642+0000","last_fullsized":"2024-04-25T21:05:35.848642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812987+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:05:35.812987+0000","last_peered":"2024-04-25T21:05:35.812987+0000","last_clean":"2024-04-25T21:05:35.812987+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:05:35.812987+0000","last_undegraded":"2024-04-25T21:05:35.812987+0000","last_fullsized":"2024-04-25T21:05:35.812987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164713+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:05:36.164713+0000","last_peered":"2024-04-25T21:05:36.164713+0000","last_clean":"2024-04-25T21:05:36.164713+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:05:36.164713+0000","last_undegraded":"2024-04-25T21:05:36.164713+0000","last_fullsized":"2024-04-25T21:05:36.164713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810024+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:05:35.810024+0000","last_peered":"2024-04-25T21:05:35.810024+0000","last_clean":"2024-04-25T21:05:35.810024+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:05:35.810024+0000","last_undegraded":"2024-04-25T21:05:35.810024+0000","last_fullsized":"2024-04-25T21:05:35.810024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164285+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:05:36.164285+0000","last_peered":"2024-04-25T21:05:36.164285+0000","last_clean":"2024-04-25T21:05:36.164285+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:05:36.164285+0000","last_undegraded":"2024-04-25T21:05:36.164285+0000","last_fullsized":"2024-04-25T21:05:36.164285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818506+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:05:35.818506+0000","last_peered":"2024-04-25T21:05:35.818506+0000","last_clean":"2024-04-25T21:05:35.818506+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:05:35.818506+0000","last_undegraded":"2024-04-25T21:05:35.818506+0000","last_fullsized":"2024-04-25T21:05:35.818506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809406+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:05:35.809406+0000","last_peered":"2024-04-25T21:05:35.809406+0000","last_clean":"2024-04-25T21:05:35.809406+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:05:35.809406+0000","last_undegraded":"2024-04-25T21:05:35.809406+0000","last_fullsized":"2024-04-25T21:05:35.809406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813376+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:05:35.813376+0000","last_peered":"2024-04-25T21:05:35.813376+0000","last_clean":"2024-04-25T21:05:35.813376+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:05:35.813376+0000","last_undegraded":"2024-04-25T21:05:35.813376+0000","last_fullsized":"2024-04-25T21:05:35.813376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246244,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1012,"num_read_kb":578,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":481464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1131282,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":480217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1073317,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":540813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":481868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":541815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":479999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072995,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:07:03.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:07:03.302 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:03.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:03 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:03 smithi098 ceph-mon[32569]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:03.471 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:03 smithi098 ceph-mon[32569]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:03.471 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:03 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:03 smithi037 ceph-mon[29195]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:03.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:03 smithi037 ceph-mon[29195]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:03.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:03 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:03 smithi177 ceph-mon[33043]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:03.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:03 smithi177 ceph-mon[33043]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:04.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:04.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:04.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:04.454 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:04.454 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:04.454 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:04.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:04.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:07:05.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 -- 172.21.15.37:0/150036335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650108a70 msgr2=0x7f1650108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:05.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 --2- 172.21.15.37:0/150036335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650108a70 0x7f1650108e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1640009990 tx=0x7f1640009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 -- 172.21.15.37:0/150036335 shutdown_connections 2024-04-25T21:07:05.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 --2- 172.21.15.37:0/150036335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 0x7f165010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 --2- 172.21.15.37:0/150036335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650108a70 0x7f1650108e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 --2- 172.21.15.37:0/150036335 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650102ac0 0x7f1650102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 -- 172.21.15.37:0/150036335 >> 172.21.15.37:0/150036335 conn(0x7f1650079bb0 msgr2=0x7f1650079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 -- 172.21.15.37:0/150036335 shutdown_connections 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 -- 172.21.15.37:0/150036335 wait complete. 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.150+0000 7f1655f48700 1 Processor -- start 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 -- start start 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 0x7f1650075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650108a70 0x7f16500757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1650112ca0 con 0x7f1650102ac0 2024-04-25T21:07:05.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1650112b20 con 0x7f1650108a70 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f1655f48700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1650112e20 con 0x7f1650103480 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33472/0 (socket says 172.21.15.37:33472) 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 -- 172.21.15.37:0/3634795873 learned_addr learned my addr 172.21.15.37:0/3634795873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164effd700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 0x7f1650075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164ffff700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650108a70 0x7f16500757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 -- 172.21.15.37:0/3634795873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 msgr2=0x7f1650075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 0x7f1650075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 -- 172.21.15.37:0/3634795873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650108a70 msgr2=0x7f16500757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650108a70 0x7f16500757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 -- 172.21.15.37:0/3634795873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16501a76c0 con 0x7f1650102ac0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.151+0000 7f164f7fe700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16400057c0 tx=0x7f16400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.152+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640027070 con 0x7f1650102ac0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.152+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16501a7950 con 0x7f1650102ac0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.152+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1640003f20 con 0x7f1650102ac0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.152+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16501a7e80 con 0x7f1650102ac0 2024-04-25T21:07:05.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.152+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1640022b80 con 0x7f1650102ac0 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.153+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c005330 con 0x7f1650102ac0 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.154+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f164002a070 con 0x7f1650102ac0 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.155+0000 7f164cff9700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 0x7f16380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.155+0000 7f164effd700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 0x7f16380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.155+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f16400d1330 con 0x7f1650102ac0 2024-04-25T21:07:05.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.155+0000 7f164effd700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 0x7f16380a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f165007a220 tx=0x7f1644009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:05.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.159+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f164005eff0 con 0x7f1650102ac0 2024-04-25T21:07:05.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.176+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1640099b60 con 0x7f1650102ac0 2024-04-25T21:07:05.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.368+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f163c000bc0 con 0x7f16380a3cd0 2024-04-25T21:07:05.369 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:05 smithi037 ceph-mon[29195]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:05.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.375+0000 7f164cff9700 1 -- 172.21.15.37:0/3634795873 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258921 (secure 0 0 0) 0x7f163c000bc0 con 0x7f16380a3cd0 2024-04-25T21:07:05.379 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 msgr2=0x7f16380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 0x7f16380a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f165007a220 tx=0x7f1644009190 comp rx=0 tx=0).stop 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 msgr2=0x7f1650074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f16400057c0 tx=0x7f16400057f0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 shutdown_connections 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16380a3cd0 0x7f16380a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1650108a70 0x7f16500757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1650102ac0 0x7f1650074d40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 --2- 172.21.15.37:0/3634795873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1650103480 0x7f1650075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:05.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 >> 172.21.15.37:0/3634795873 conn(0x7f1650079bb0 msgr2=0x7f16501017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:05.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 shutdown_connections 2024-04-25T21:07:05.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:05.378+0000 7f1655f48700 1 -- 172.21.15.37:0/3634795873 wait complete. 2024-04-25T21:07:05.382 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:07:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:05 smithi177 ceph-mon[33043]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:05 smithi098 ceph-mon[32569]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:06.091 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-25T21:07:04.227197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1188,"num_read_kb":722,"num_write":427,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361332,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580492,"statfs":{"total":1151940427776,"available":1151570423808,"internally_reserved":0,"allocated":12435456,"data_stored":7724292,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003302"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812561+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:05:35.812561+0000","last_peered":"2024-04-25T21:05:35.812561+0000","last_clean":"2024-04-25T21:05:35.812561+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:05:35.812561+0000","last_undegraded":"2024-04-25T21:05:35.812561+0000","last_fullsized":"2024-04-25T21:05:35.812561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812177+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:05:35.812177+0000","last_peered":"2024-04-25T21:05:35.812177+0000","last_clean":"2024-04-25T21:05:35.812177+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:05:35.812177+0000","last_undegraded":"2024-04-25T21:05:35.812177+0000","last_fullsized":"2024-04-25T21:05:35.812177+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811939+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:05:35.811939+0000","last_peered":"2024-04-25T21:05:35.811939+0000","last_clean":"2024-04-25T21:05:35.811939+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:05:35.811939+0000","last_undegraded":"2024-04-25T21:05:35.811939+0000","last_fullsized":"2024-04-25T21:05:35.811939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280187+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:05:36.280187+0000","last_peered":"2024-04-25T21:05:36.280187+0000","last_clean":"2024-04-25T21:05:36.280187+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:05:36.280187+0000","last_undegraded":"2024-04-25T21:05:36.280187+0000","last_fullsized":"2024-04-25T21:05:36.280187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815068+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:05:35.815068+0000","last_peered":"2024-04-25T21:05:35.815068+0000","last_clean":"2024-04-25T21:05:35.815068+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:05:35.815068+0000","last_undegraded":"2024-04-25T21:05:35.815068+0000","last_fullsized":"2024-04-25T21:05:35.815068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054645+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:05:36.054645+0000","last_peered":"2024-04-25T21:05:36.054645+0000","last_clean":"2024-04-25T21:05:36.054645+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:05:36.054645+0000","last_undegraded":"2024-04-25T21:05:36.054645+0000","last_fullsized":"2024-04-25T21:05:36.054645+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810131+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:05:35.810131+0000","last_peered":"2024-04-25T21:05:35.810131+0000","last_clean":"2024-04-25T21:05:35.810131+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:05:35.810131+0000","last_undegraded":"2024-04-25T21:05:35.810131+0000","last_fullsized":"2024-04-25T21:05:35.810131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164368+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:05:36.164368+0000","last_peered":"2024-04-25T21:05:36.164368+0000","last_clean":"2024-04-25T21:05:36.164368+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:05:36.164368+0000","last_undegraded":"2024-04-25T21:05:36.164368+0000","last_fullsized":"2024-04-25T21:05:36.164368+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812458+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:05:35.812458+0000","last_peered":"2024-04-25T21:05:35.812458+0000","last_clean":"2024-04-25T21:05:35.812458+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:05:35.812458+0000","last_undegraded":"2024-04-25T21:05:35.812458+0000","last_fullsized":"2024-04-25T21:05:35.812458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054978+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:05:36.054978+0000","last_peered":"2024-04-25T21:05:36.054978+0000","last_clean":"2024-04-25T21:05:36.054978+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:05:36.054978+0000","last_undegraded":"2024-04-25T21:05:36.054978+0000","last_fullsized":"2024-04-25T21:05:36.054978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810362+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:05:35.810362+0000","last_peered":"2024-04-25T21:05:35.810362+0000","last_clean":"2024-04-25T21:05:35.810362+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:05:35.810362+0000","last_undegraded":"2024-04-25T21:05:35.810362+0000","last_fullsized":"2024-04-25T21:05:35.810362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809516+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:05:35.809516+0000","last_peered":"2024-04-25T21:05:35.809516+0000","last_clean":"2024-04-25T21:05:35.809516+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:05:35.809516+0000","last_undegraded":"2024-04-25T21:05:35.809516+0000","last_fullsized":"2024-04-25T21:05:35.809516+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811795+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:05:35.811795+0000","last_peered":"2024-04-25T21:05:35.811795+0000","last_clean":"2024-04-25T21:05:35.811795+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:05:35.811795+0000","last_undegraded":"2024-04-25T21:05:35.811795+0000","last_fullsized":"2024-04-25T21:05:35.811795+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818318+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:05:35.818318+0000","last_peered":"2024-04-25T21:05:35.818318+0000","last_clean":"2024-04-25T21:05:35.818318+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:05:35.818318+0000","last_undegraded":"2024-04-25T21:05:35.818318+0000","last_fullsized":"2024-04-25T21:05:35.818318+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811748+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:05:35.811748+0000","last_peered":"2024-04-25T21:05:35.811748+0000","last_clean":"2024-04-25T21:05:35.811748+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:05:35.811748+0000","last_undegraded":"2024-04-25T21:05:35.811748+0000","last_fullsized":"2024-04-25T21:05:35.811748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812733+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:05:35.812733+0000","last_peered":"2024-04-25T21:05:35.812733+0000","last_clean":"2024-04-25T21:05:35.812733+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:05:35.812733+0000","last_undegraded":"2024-04-25T21:05:35.812733+0000","last_fullsized":"2024-04-25T21:05:35.812733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847351+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:05:35.847351+0000","last_peered":"2024-04-25T21:05:35.847351+0000","last_clean":"2024-04-25T21:05:35.847351+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:05:35.847351+0000","last_undegraded":"2024-04-25T21:05:35.847351+0000","last_fullsized":"2024-04-25T21:05:35.847351+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813305+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:05:35.813305+0000","last_peered":"2024-04-25T21:05:35.813305+0000","last_clean":"2024-04-25T21:05:35.813305+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:05:35.813305+0000","last_undegraded":"2024-04-25T21:05:35.813305+0000","last_fullsized":"2024-04-25T21:05:35.813305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:07:03.394386+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:07:03.394386+0000","last_peered":"2024-04-25T21:07:03.394386+0000","last_clean":"2024-04-25T21:07:03.394386+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:07:03.394386+0000","last_undegraded":"2024-04-25T21:07:03.394386+0000","last_fullsized":"2024-04-25T21:07:03.394386+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164650+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:05:36.164650+0000","last_peered":"2024-04-25T21:05:36.164650+0000","last_clean":"2024-04-25T21:05:36.164650+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:05:36.164650+0000","last_undegraded":"2024-04-25T21:05:36.164650+0000","last_fullsized":"2024-04-25T21:05:36.164650+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164841+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:05:36.164841+0000","last_peered":"2024-04-25T21:05:36.164841+0000","last_clean":"2024-04-25T21:05:36.164841+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:05:36.164841+0000","last_undegraded":"2024-04-25T21:05:36.164841+0000","last_fullsized":"2024-04-25T21:05:36.164841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812294+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:05:35.812294+0000","last_peered":"2024-04-25T21:05:35.812294+0000","last_clean":"2024-04-25T21:05:35.812294+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:05:35.812294+0000","last_undegraded":"2024-04-25T21:05:35.812294+0000","last_fullsized":"2024-04-25T21:05:35.812294+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815926+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:05:35.815926+0000","last_peered":"2024-04-25T21:05:35.815926+0000","last_clean":"2024-04-25T21:05:35.815926+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:05:35.815926+0000","last_undegraded":"2024-04-25T21:05:35.815926+0000","last_fullsized":"2024-04-25T21:05:35.815926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811808+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:05:35.811808+0000","last_peered":"2024-04-25T21:05:35.811808+0000","last_clean":"2024-04-25T21:05:35.811808+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:05:35.811808+0000","last_undegraded":"2024-04-25T21:05:35.811808+0000","last_fullsized":"2024-04-25T21:05:35.811808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.823684+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:05:35.823684+0000","last_peered":"2024-04-25T21:05:35.823684+0000","last_clean":"2024-04-25T21:05:35.823684+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:05:35.823684+0000","last_undegraded":"2024-04-25T21:05:35.823684+0000","last_fullsized":"2024-04-25T21:05:35.823684+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810659+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:05:35.810659+0000","last_peered":"2024-04-25T21:05:35.810659+0000","last_clean":"2024-04-25T21:05:35.810659+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:05:35.810659+0000","last_undegraded":"2024-04-25T21:05:35.810659+0000","last_fullsized":"2024-04-25T21:05:35.810659+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:58.389544+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:06:58.389544+0000","last_peered":"2024-04-25T21:06:58.389544+0000","last_clean":"2024-04-25T21:06:58.389544+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:06:58.389544+0000","last_undegraded":"2024-04-25T21:06:58.389544+0000","last_fullsized":"2024-04-25T21:06:58.389544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815987+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:05:35.815987+0000","last_peered":"2024-04-25T21:05:35.815987+0000","last_clean":"2024-04-25T21:05:35.815987+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:05:35.815987+0000","last_undegraded":"2024-04-25T21:05:35.815987+0000","last_fullsized":"2024-04-25T21:05:35.815987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054563+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:05:36.054563+0000","last_peered":"2024-04-25T21:05:36.054563+0000","last_clean":"2024-04-25T21:05:36.054563+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:05:36.054563+0000","last_undegraded":"2024-04-25T21:05:36.054563+0000","last_fullsized":"2024-04-25T21:05:36.054563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810324+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:05:35.810324+0000","last_peered":"2024-04-25T21:05:35.810324+0000","last_clean":"2024-04-25T21:05:35.810324+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:05:35.810324+0000","last_undegraded":"2024-04-25T21:05:35.810324+0000","last_fullsized":"2024-04-25T21:05:35.810324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809323+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:05:35.809323+0000","last_peered":"2024-04-25T21:05:35.809323+0000","last_clean":"2024-04-25T21:05:35.809323+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:05:35.809323+0000","last_undegraded":"2024-04-25T21:05:35.809323+0000","last_fullsized":"2024-04-25T21:05:35.809323+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164264+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:05:36.164264+0000","last_peered":"2024-04-25T21:05:36.164264+0000","last_clean":"2024-04-25T21:05:36.164264+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:05:36.164264+0000","last_undegraded":"2024-04-25T21:05:36.164264+0000","last_fullsized":"2024-04-25T21:05:36.164264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809991+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:05:35.809991+0000","last_peered":"2024-04-25T21:05:35.809991+0000","last_clean":"2024-04-25T21:05:35.809991+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:05:35.809991+0000","last_undegraded":"2024-04-25T21:05:35.809991+0000","last_fullsized":"2024-04-25T21:05:35.809991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847028+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:05:35.847028+0000","last_peered":"2024-04-25T21:05:35.847028+0000","last_clean":"2024-04-25T21:05:35.847028+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:05:35.847028+0000","last_undegraded":"2024-04-25T21:05:35.847028+0000","last_fullsized":"2024-04-25T21:05:35.847028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:07:00.391311+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:07:00.391311+0000","last_peered":"2024-04-25T21:07:00.391311+0000","last_clean":"2024-04-25T21:07:00.391311+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:07:00.391311+0000","last_undegraded":"2024-04-25T21:07:00.391311+0000","last_fullsized":"2024-04-25T21:07:00.391311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812371+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:05:35.812371+0000","last_peered":"2024-04-25T21:05:35.812371+0000","last_clean":"2024-04-25T21:05:35.812371+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:05:35.812371+0000","last_undegraded":"2024-04-25T21:05:35.812371+0000","last_fullsized":"2024-04-25T21:05:35.812371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810524+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:05:35.810524+0000","last_peered":"2024-04-25T21:05:35.810524+0000","last_clean":"2024-04-25T21:05:35.810524+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:05:35.810524+0000","last_undegraded":"2024-04-25T21:05:35.810524+0000","last_fullsized":"2024-04-25T21:05:35.810524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812286+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:05:35.812286+0000","last_peered":"2024-04-25T21:05:35.812286+0000","last_clean":"2024-04-25T21:05:35.812286+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:05:35.812286+0000","last_undegraded":"2024-04-25T21:05:35.812286+0000","last_fullsized":"2024-04-25T21:05:35.812286+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810988+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:05:35.810988+0000","last_peered":"2024-04-25T21:05:35.810988+0000","last_clean":"2024-04-25T21:05:35.810988+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:05:35.810988+0000","last_undegraded":"2024-04-25T21:05:35.810988+0000","last_fullsized":"2024-04-25T21:05:35.810988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812018+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:05:35.812018+0000","last_peered":"2024-04-25T21:05:35.812018+0000","last_clean":"2024-04-25T21:05:35.812018+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:05:35.812018+0000","last_undegraded":"2024-04-25T21:05:35.812018+0000","last_fullsized":"2024-04-25T21:05:35.812018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":267,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:07:00.404128+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:07:00.404128+0000","last_peered":"2024-04-25T21:07:00.404128+0000","last_clean":"2024-04-25T21:07:00.404128+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:07:00.404128+0000","last_undegraded":"2024-04-25T21:07:00.404128+0000","last_fullsized":"2024-04-25T21:07:00.404128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":163,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815665+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:05:35.815665+0000","last_peered":"2024-04-25T21:05:35.815665+0000","last_clean":"2024-04-25T21:05:35.815665+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:05:35.815665+0000","last_undegraded":"2024-04-25T21:05:35.815665+0000","last_fullsized":"2024-04-25T21:05:35.815665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813128+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:05:35.813128+0000","last_peered":"2024-04-25T21:05:35.813128+0000","last_clean":"2024-04-25T21:05:35.813128+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:05:35.813128+0000","last_undegraded":"2024-04-25T21:05:35.813128+0000","last_fullsized":"2024-04-25T21:05:35.813128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809221+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:05:35.809221+0000","last_peered":"2024-04-25T21:05:35.809221+0000","last_clean":"2024-04-25T21:05:35.809221+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:05:35.809221+0000","last_undegraded":"2024-04-25T21:05:35.809221+0000","last_fullsized":"2024-04-25T21:05:35.809221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810239+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:05:35.810239+0000","last_peered":"2024-04-25T21:05:35.810239+0000","last_clean":"2024-04-25T21:05:35.810239+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:05:35.810239+0000","last_undegraded":"2024-04-25T21:05:35.810239+0000","last_fullsized":"2024-04-25T21:05:35.810239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848492+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:05:35.848492+0000","last_peered":"2024-04-25T21:05:35.848492+0000","last_clean":"2024-04-25T21:05:35.848492+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:05:35.848492+0000","last_undegraded":"2024-04-25T21:05:35.848492+0000","last_fullsized":"2024-04-25T21:05:35.848492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":134,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:59.390009+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:06:59.390009+0000","last_peered":"2024-04-25T21:06:59.390009+0000","last_clean":"2024-04-25T21:06:59.390009+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:06:59.390009+0000","last_undegraded":"2024-04-25T21:06:59.390009+0000","last_fullsized":"2024-04-25T21:06:59.390009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809971+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:05:35.809971+0000","last_peered":"2024-04-25T21:05:35.809971+0000","last_clean":"2024-04-25T21:05:35.809971+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:05:35.809971+0000","last_undegraded":"2024-04-25T21:05:35.809971+0000","last_fullsized":"2024-04-25T21:05:35.809971+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815899+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:05:35.815899+0000","last_peered":"2024-04-25T21:05:35.815899+0000","last_clean":"2024-04-25T21:05:35.815899+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:05:35.815899+0000","last_undegraded":"2024-04-25T21:05:35.815899+0000","last_fullsized":"2024-04-25T21:05:35.815899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810151+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:05:35.810151+0000","last_peered":"2024-04-25T21:05:35.810151+0000","last_clean":"2024-04-25T21:05:35.810151+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:05:35.810151+0000","last_undegraded":"2024-04-25T21:05:35.810151+0000","last_fullsized":"2024-04-25T21:05:35.810151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815284+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:05:35.815284+0000","last_peered":"2024-04-25T21:05:35.815284+0000","last_clean":"2024-04-25T21:05:35.815284+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:05:35.815284+0000","last_undegraded":"2024-04-25T21:05:35.815284+0000","last_fullsized":"2024-04-25T21:05:35.815284+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812340+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:05:35.812340+0000","last_peered":"2024-04-25T21:05:35.812340+0000","last_clean":"2024-04-25T21:05:35.812340+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:05:35.812340+0000","last_undegraded":"2024-04-25T21:05:35.812340+0000","last_fullsized":"2024-04-25T21:05:35.812340+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.847267+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:05:35.847267+0000","last_peered":"2024-04-25T21:05:35.847267+0000","last_clean":"2024-04-25T21:05:35.847267+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:05:35.847267+0000","last_undegraded":"2024-04-25T21:05:35.847267+0000","last_fullsized":"2024-04-25T21:05:35.847267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054796+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:05:36.054796+0000","last_peered":"2024-04-25T21:05:36.054796+0000","last_clean":"2024-04-25T21:05:36.054796+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:05:36.054796+0000","last_undegraded":"2024-04-25T21:05:36.054796+0000","last_fullsized":"2024-04-25T21:05:36.054796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:59.390009+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:06:59.390009+0000","last_peered":"2024-04-25T21:06:59.390009+0000","last_clean":"2024-04-25T21:06:59.390009+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:06:59.390009+0000","last_undegraded":"2024-04-25T21:06:59.390009+0000","last_fullsized":"2024-04-25T21:06:59.390009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.054640+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:05:36.054640+0000","last_peered":"2024-04-25T21:05:36.054640+0000","last_clean":"2024-04-25T21:05:36.054640+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:05:36.054640+0000","last_undegraded":"2024-04-25T21:05:36.054640+0000","last_fullsized":"2024-04-25T21:05:36.054640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810247+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:05:35.810247+0000","last_peered":"2024-04-25T21:05:35.810247+0000","last_clean":"2024-04-25T21:05:35.810247+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:05:35.810247+0000","last_undegraded":"2024-04-25T21:05:35.810247+0000","last_fullsized":"2024-04-25T21:05:35.810247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:07:02.393210+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:07:02.393210+0000","last_peered":"2024-04-25T21:07:02.393210+0000","last_clean":"2024-04-25T21:07:02.393210+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:07:02.393210+0000","last_undegraded":"2024-04-25T21:07:02.393210+0000","last_fullsized":"2024-04-25T21:07:02.393210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280055+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:05:36.280055+0000","last_peered":"2024-04-25T21:05:36.280055+0000","last_clean":"2024-04-25T21:05:36.280055+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:05:36.280055+0000","last_undegraded":"2024-04-25T21:05:36.280055+0000","last_fullsized":"2024-04-25T21:05:36.280055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810193+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:05:35.810193+0000","last_peered":"2024-04-25T21:05:35.810193+0000","last_clean":"2024-04-25T21:05:35.810193+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:05:35.810193+0000","last_undegraded":"2024-04-25T21:05:35.810193+0000","last_fullsized":"2024-04-25T21:05:35.810193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809159+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:05:35.809159+0000","last_peered":"2024-04-25T21:05:35.809159+0000","last_clean":"2024-04-25T21:05:35.809159+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:05:35.809159+0000","last_undegraded":"2024-04-25T21:05:35.809159+0000","last_fullsized":"2024-04-25T21:05:35.809159+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810291+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:05:35.810291+0000","last_peered":"2024-04-25T21:05:35.810291+0000","last_clean":"2024-04-25T21:05:35.810291+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:05:35.810291+0000","last_undegraded":"2024-04-25T21:05:35.810291+0000","last_fullsized":"2024-04-25T21:05:35.810291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815800+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:05:35.815800+0000","last_peered":"2024-04-25T21:05:35.815800+0000","last_clean":"2024-04-25T21:05:35.815800+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:05:35.815800+0000","last_undegraded":"2024-04-25T21:05:35.815800+0000","last_fullsized":"2024-04-25T21:05:35.815800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809414+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:05:35.809414+0000","last_peered":"2024-04-25T21:05:35.809414+0000","last_clean":"2024-04-25T21:05:35.809414+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:05:35.809414+0000","last_undegraded":"2024-04-25T21:05:35.809414+0000","last_fullsized":"2024-04-25T21:05:35.809414+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'15","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:41.673824+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:05:41.673824+0000","last_peered":"2024-04-25T21:05:41.673824+0000","last_clean":"2024-04-25T21:05:41.673824+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:05:41.673824+0000","last_undegraded":"2024-04-25T21:05:41.673824+0000","last_fullsized":"2024-04-25T21:05:41.673824+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164820+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:05:36.164820+0000","last_peered":"2024-04-25T21:05:36.164820+0000","last_clean":"2024-04-25T21:05:36.164820+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:05:36.164820+0000","last_undegraded":"2024-04-25T21:05:36.164820+0000","last_fullsized":"2024-04-25T21:05:36.164820+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.280051+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:05:36.280051+0000","last_peered":"2024-04-25T21:05:36.280051+0000","last_clean":"2024-04-25T21:05:36.280051+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:05:36.280051+0000","last_undegraded":"2024-04-25T21:05:36.280051+0000","last_fullsized":"2024-04-25T21:05:36.280051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812196+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:05:35.812196+0000","last_peered":"2024-04-25T21:05:35.812196+0000","last_clean":"2024-04-25T21:05:35.812196+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:05:35.812196+0000","last_undegraded":"2024-04-25T21:05:35.812196+0000","last_fullsized":"2024-04-25T21:05:35.812196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.820411+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:05:35.820411+0000","last_peered":"2024-04-25T21:05:35.820411+0000","last_clean":"2024-04-25T21:05:35.820411+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:05:35.820411+0000","last_undegraded":"2024-04-25T21:05:35.820411+0000","last_fullsized":"2024-04-25T21:05:35.820411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.816311+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:05:35.816311+0000","last_peered":"2024-04-25T21:05:35.816311+0000","last_clean":"2024-04-25T21:05:35.816311+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:05:35.816311+0000","last_undegraded":"2024-04-25T21:05:35.816311+0000","last_fullsized":"2024-04-25T21:05:35.816311+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164359+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:05:36.164359+0000","last_peered":"2024-04-25T21:05:36.164359+0000","last_clean":"2024-04-25T21:05:36.164359+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:05:36.164359+0000","last_undegraded":"2024-04-25T21:05:36.164359+0000","last_fullsized":"2024-04-25T21:05:36.164359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813860+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:05:35.813860+0000","last_peered":"2024-04-25T21:05:35.813860+0000","last_clean":"2024-04-25T21:05:35.813860+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:05:35.813860+0000","last_undegraded":"2024-04-25T21:05:35.813860+0000","last_fullsized":"2024-04-25T21:05:35.813860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812840+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:05:35.812840+0000","last_peered":"2024-04-25T21:05:35.812840+0000","last_clean":"2024-04-25T21:05:35.812840+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:05:35.812840+0000","last_undegraded":"2024-04-25T21:05:35.812840+0000","last_fullsized":"2024-04-25T21:05:35.812840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164520+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:05:36.164520+0000","last_peered":"2024-04-25T21:05:36.164520+0000","last_clean":"2024-04-25T21:05:36.164520+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:05:36.164520+0000","last_undegraded":"2024-04-25T21:05:36.164520+0000","last_fullsized":"2024-04-25T21:05:36.164520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.815193+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:05:35.815193+0000","last_peered":"2024-04-25T21:05:35.815193+0000","last_clean":"2024-04-25T21:05:35.815193+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:05:35.815193+0000","last_undegraded":"2024-04-25T21:05:35.815193+0000","last_fullsized":"2024-04-25T21:05:35.815193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810529+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:05:35.810529+0000","last_peered":"2024-04-25T21:05:35.810529+0000","last_clean":"2024-04-25T21:05:35.810529+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:05:35.810529+0000","last_undegraded":"2024-04-25T21:05:35.810529+0000","last_fullsized":"2024-04-25T21:05:35.810529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:06:58.402134+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:06:58.402134+0000","last_peered":"2024-04-25T21:06:58.402134+0000","last_clean":"2024-04-25T21:06:58.402134+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:06:58.402134+0000","last_undegraded":"2024-04-25T21:06:58.402134+0000","last_fullsized":"2024-04-25T21:06:58.402134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810542+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:05:35.810542+0000","last_peered":"2024-04-25T21:05:35.810542+0000","last_clean":"2024-04-25T21:05:35.810542+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:05:35.810542+0000","last_undegraded":"2024-04-25T21:05:35.810542+0000","last_fullsized":"2024-04-25T21:05:35.810542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810498+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:05:35.810498+0000","last_peered":"2024-04-25T21:05:35.810498+0000","last_clean":"2024-04-25T21:05:35.810498+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:05:35.810498+0000","last_undegraded":"2024-04-25T21:05:35.810498+0000","last_fullsized":"2024-04-25T21:05:35.810498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810354+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:05:35.810354+0000","last_peered":"2024-04-25T21:05:35.810354+0000","last_clean":"2024-04-25T21:05:35.810354+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:05:35.810354+0000","last_undegraded":"2024-04-25T21:05:35.810354+0000","last_fullsized":"2024-04-25T21:05:35.810354+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809534+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:05:35.809534+0000","last_peered":"2024-04-25T21:05:35.809534+0000","last_clean":"2024-04-25T21:05:35.809534+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:05:35.809534+0000","last_undegraded":"2024-04-25T21:05:35.809534+0000","last_fullsized":"2024-04-25T21:05:35.809534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810657+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:05:35.810657+0000","last_peered":"2024-04-25T21:05:35.810657+0000","last_clean":"2024-04-25T21:05:35.810657+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:05:35.810657+0000","last_undegraded":"2024-04-25T21:05:35.810657+0000","last_fullsized":"2024-04-25T21:05:35.810657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848543+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:05:35.848543+0000","last_peered":"2024-04-25T21:05:35.848543+0000","last_clean":"2024-04-25T21:05:35.848543+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:05:35.848543+0000","last_undegraded":"2024-04-25T21:05:35.848543+0000","last_fullsized":"2024-04-25T21:05:35.848543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.814755+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:05:35.814755+0000","last_peered":"2024-04-25T21:05:35.814755+0000","last_clean":"2024-04-25T21:05:35.814755+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:05:35.814755+0000","last_undegraded":"2024-04-25T21:05:35.814755+0000","last_fullsized":"2024-04-25T21:05:35.814755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812713+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:05:35.812713+0000","last_peered":"2024-04-25T21:05:35.812713+0000","last_clean":"2024-04-25T21:05:35.812713+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:05:35.812713+0000","last_undegraded":"2024-04-25T21:05:35.812713+0000","last_fullsized":"2024-04-25T21:05:35.812713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.811944+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:05:35.811944+0000","last_peered":"2024-04-25T21:05:35.811944+0000","last_clean":"2024-04-25T21:05:35.811944+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:05:35.811944+0000","last_undegraded":"2024-04-25T21:05:35.811944+0000","last_fullsized":"2024-04-25T21:05:35.811944+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809423+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:05:35.809423+0000","last_peered":"2024-04-25T21:05:35.809423+0000","last_clean":"2024-04-25T21:05:35.809423+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:05:35.809423+0000","last_undegraded":"2024-04-25T21:05:35.809423+0000","last_fullsized":"2024-04-25T21:05:35.809423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.848642+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:05:35.848642+0000","last_peered":"2024-04-25T21:05:35.848642+0000","last_clean":"2024-04-25T21:05:35.848642+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:05:35.848642+0000","last_undegraded":"2024-04-25T21:05:35.848642+0000","last_fullsized":"2024-04-25T21:05:35.848642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.812987+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:05:35.812987+0000","last_peered":"2024-04-25T21:05:35.812987+0000","last_clean":"2024-04-25T21:05:35.812987+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:05:35.812987+0000","last_undegraded":"2024-04-25T21:05:35.812987+0000","last_fullsized":"2024-04-25T21:05:35.812987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164713+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:05:36.164713+0000","last_peered":"2024-04-25T21:05:36.164713+0000","last_clean":"2024-04-25T21:05:36.164713+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:05:36.164713+0000","last_undegraded":"2024-04-25T21:05:36.164713+0000","last_fullsized":"2024-04-25T21:05:36.164713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.810024+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:05:35.810024+0000","last_peered":"2024-04-25T21:05:35.810024+0000","last_clean":"2024-04-25T21:05:35.810024+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:05:35.810024+0000","last_undegraded":"2024-04-25T21:05:35.810024+0000","last_fullsized":"2024-04-25T21:05:35.810024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:36.164285+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:05:36.164285+0000","last_peered":"2024-04-25T21:05:36.164285+0000","last_clean":"2024-04-25T21:05:36.164285+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:05:36.164285+0000","last_undegraded":"2024-04-25T21:05:36.164285+0000","last_fullsized":"2024-04-25T21:05:36.164285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.818506+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:05:35.818506+0000","last_peered":"2024-04-25T21:05:35.818506+0000","last_clean":"2024-04-25T21:05:35.818506+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:05:35.818506+0000","last_undegraded":"2024-04-25T21:05:35.818506+0000","last_fullsized":"2024-04-25T21:05:35.818506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.809406+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:05:35.809406+0000","last_peered":"2024-04-25T21:05:35.809406+0000","last_clean":"2024-04-25T21:05:35.809406+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:05:35.809406+0000","last_undegraded":"2024-04-25T21:05:35.809406+0000","last_fullsized":"2024-04-25T21:05:35.809406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T21:05:35.813376+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:05:35.813376+0000","last_peered":"2024-04-25T21:05:35.813376+0000","last_clean":"2024-04-25T21:05:35.813376+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:05:35.813376+0000","last_undegraded":"2024-04-25T21:05:35.813376+0000","last_fullsized":"2024-04-25T21:05:35.813376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246244,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1052,"num_read_kb":603,"num_write":244,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":481464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1131282,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":480217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":1073317,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":540813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":481868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":541815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":479999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072995,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34007,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:07:06.096 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T21:07:06.096 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T21:07:06.096 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T21:07:06.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T21:07:06.349 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:06.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:06 smithi037 ceph-mon[29195]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T21:07:06.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:06 smithi037 ceph-mon[29195]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:06.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:06 smithi177 ceph-mon[33043]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T21:07:06.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:06 smithi177 ceph-mon[33043]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:06.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:06 smithi098 ceph-mon[32569]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T21:07:06.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:06 smithi098 ceph-mon[32569]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:07:07.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:07 smithi037 ceph-mon[29195]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:07.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:07 smithi177 ceph-mon[33043]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:07 smithi098 ceph-mon[32569]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/3560737195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 msgr2=0x7f0f18108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/3560737195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f18108e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009990 tx=0x7f0f08009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/3560737195 shutdown_connections 2024-04-25T21:07:08.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/3560737195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806b0f0 0x7f0f1810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/3560737195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f18108e10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/3560737195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806a750 0x7f0f1806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/3560737195 >> 172.21.15.37:0/3560737195 conn(0x7f0f180788f0 msgr2=0x7f0f18078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/3560737195 shutdown_connections 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.060+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/3560737195 wait complete. 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 Processor -- start 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 -- start start 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806a750 0x7f0f181990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 0x7f0f18199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18112bb0 con 0x7f0f18108a30 2024-04-25T21:07:08.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18112a30 con 0x7f0f1806b0f0 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f1d2c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f18112d30 con 0x7f0f1806a750 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f177fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33484/0 (socket says 172.21.15.37:33484) 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f177fe700 1 -- 172.21.15.37:0/885944366 learned_addr learned my addr 172.21.15.37:0/885944366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.061+0000 7f0f16ffd700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806a750 0x7f0f181990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f167fc700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 0x7f0f18199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 -- 172.21.15.37:0/885944366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806a750 msgr2=0x7f0f181990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806a750 0x7f0f181990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 -- 172.21.15.37:0/885944366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 msgr2=0x7f0f18199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 0x7f0f18199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 -- 172.21.15.37:0/885944366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1819e0d0 con 0x7f0f18108a30 2024-04-25T21:07:08.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f167fc700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 0x7f0f18199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f177fe700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f10007d70 tx=0x7f0f1000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f10016020 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1819e3c0 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f1000ad40 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f100115e0 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.062+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a5c40 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.063+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f18069730 con 0x7f0f18108a30 2024-04-25T21:07:08.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.064+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f10019020 con 0x7f0f18108a30 2024-04-25T21:07:08.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.065+0000 7f0efffff700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:08.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.065+0000 7f0f16ffd700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:08.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.065+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0f100d03c0 con 0x7f0f18108a30 2024-04-25T21:07:08.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.065+0000 7f0f16ffd700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 0x7f0f000a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b5c0 tx=0x7f0f08005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:08.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.069+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f10020020 con 0x7f0f18108a30 2024-04-25T21:07:08.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.328+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0f1802df80 con 0x7f0f18108a30 2024-04-25T21:07:08.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.328+0000 7f0efffff700 1 -- 172.21.15.37:0/885944366 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0f1009cd50 con 0x7f0f18108a30 2024-04-25T21:07:08.329 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:08.329 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 msgr2=0x7f0f000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 0x7f0f000a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b5c0 tx=0x7f0f08005960 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 msgr2=0x7f0f1819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f10007d70 tx=0x7f0f1000a340 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 shutdown_connections 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f18108a30 0x7f0f1819d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1806a750 0x7f0f181990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f000a3c40 0x7f0f000a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 --2- 172.21.15.37:0/885944366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f1806b0f0 0x7f0f18199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 >> 172.21.15.37:0/885944366 conn(0x7f0f180788f0 msgr2=0x7f0f180ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 shutdown_connections 2024-04-25T21:07:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:08.330+0000 7f0f1d2c5700 1 -- 172.21.15.37:0/885944366 wait complete. 2024-04-25T21:07:09.105 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T21:07:09.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:07:09.363 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:09.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:09 smithi037 ceph-mon[29195]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:09.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:09 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/885944366' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T21:07:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:09 smithi177 ceph-mon[33043]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:09.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:09 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/885944366' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T21:07:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:09 smithi098 ceph-mon[32569]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:09 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/885944366' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T21:07:11.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 -- 172.21.15.37:0/103955305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa84108a60 msgr2=0x7faa84108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:11.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 --2- 172.21.15.37:0/103955305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa84108a60 0x7faa84108e40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa80009990 tx=0x7faa80009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 -- 172.21.15.37:0/103955305 shutdown_connections 2024-04-25T21:07:11.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 --2- 172.21.15.37:0/103955305 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa8406bbb0 0x7faa8410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 --2- 172.21.15.37:0/103955305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa84108a60 0x7faa84108e40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 --2- 172.21.15.37:0/103955305 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa8406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 -- 172.21.15.37:0/103955305 >> 172.21.15.37:0/103955305 conn(0x7faa840789e0 msgr2=0x7faa84078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 -- 172.21.15.37:0/103955305 shutdown_connections 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 -- 172.21.15.37:0/103955305 wait complete. 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.137+0000 7faa8b715700 1 Processor -- start 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 -- start start 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa841019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa84108a60 0x7faa84102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faa84112ca0 con 0x7faa8406bbb0 2024-04-25T21:07:11.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa84112b20 con 0x7faa8406b1f0 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa8b715700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faa84112e20 con 0x7faa84108a60 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60456/0 (socket says 172.21.15.37:60456) 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 -- 172.21.15.37:0/271489673 learned_addr learned my addr 172.21.15.37:0/271489673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa894b1700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa841019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa89cb2700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa84108a60 0x7faa84102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 -- 172.21.15.37:0/271489673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa84108a60 msgr2=0x7faa84102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa84108a60 0x7faa84102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 -- 172.21.15.37:0/271489673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 msgr2=0x7faa841019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa841019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 -- 172.21.15.37:0/271489673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa841a76d0 con 0x7faa8406bbb0 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa894b1700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa841019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:11.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.138+0000 7faa88cb0700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c920 tx=0x7faa7400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:11.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.139+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74007b00 con 0x7faa8406bbb0 2024-04-25T21:07:11.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.139+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa841a79c0 con 0x7faa8406bbb0 2024-04-25T21:07:11.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.139+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa74004ce0 con 0x7faa8406bbb0 2024-04-25T21:07:11.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.139+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74005650 con 0x7faa8406bbb0 2024-04-25T21:07:11.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.139+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa841a7f00 con 0x7faa8406bbb0 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.140+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8410b060 con 0x7faa8406bbb0 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.141+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa74007ca0 con 0x7faa8406bbb0 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.141+0000 7faa7a7fc700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 0x7faa700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.141+0000 7faa894b1700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 0x7faa700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.142+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7faa740d0380 con 0x7faa8406bbb0 2024-04-25T21:07:11.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.142+0000 7faa894b1700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 0x7faa700a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa8000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:11.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.146+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa74014030 con 0x7faa8406bbb0 2024-04-25T21:07:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.400+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa84069730 con 0x7faa8406bbb0 2024-04-25T21:07:11.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:11 smithi037 ceph-mon[29195]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.401+0000 7faa7a7fc700 1 -- 172.21.15.37:0/271489673 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7faa7409cd70 con 0x7faa8406bbb0 2024-04-25T21:07:11.402 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:11.402 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:07:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.403+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 msgr2=0x7faa700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 0x7faa700a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faa80006010 tx=0x7faa8000b540 comp rx=0 tx=0).stop 2024-04-25T21:07:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 msgr2=0x7faa84101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c920 tx=0x7faa7400cce0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 shutdown_connections 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faa84108a60 0x7faa84102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faa700a3c40 0x7faa700a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa8406b1f0 0x7faa841019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 --2- 172.21.15.37:0/271489673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa8406bbb0 0x7faa84101ef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 >> 172.21.15.37:0/271489673 conn(0x7faa840789e0 msgr2=0x7faa840ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 shutdown_connections 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:11.404+0000 7faa8b715700 1 -- 172.21.15.37:0/271489673 wait complete. 2024-04-25T21:07:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:07:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:11 smithi177 ceph-mon[33043]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:11 smithi098 ceph-mon[32569]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:12.136 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:07:12.389 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:12.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:12 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/271489673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:12.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:12 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/271489673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:12.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:12 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/271489673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:13.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:13 smithi177 ceph-mon[33043]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:13.612 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:13 smithi037 ceph-mon[29195]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:13.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:13 smithi098 ceph-mon[32569]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 -- 172.21.15.37:0/567067290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006b1f0 msgr2=0x7fbed006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 --2- 172.21.15.37:0/567067290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006b1f0 0x7fbed006b670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009990 tx=0x7fbeb8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 -- 172.21.15.37:0/567067290 shutdown_connections 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 --2- 172.21.15.37:0/567067290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed006bbb0 0x7fbed010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 --2- 172.21.15.37:0/567067290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed0108a60 0x7fbed0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 --2- 172.21.15.37:0/567067290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006b1f0 0x7fbed006b670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 -- 172.21.15.37:0/567067290 >> 172.21.15.37:0/567067290 conn(0x7fbed00789e0 msgr2=0x7fbed0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 -- 172.21.15.37:0/567067290 shutdown_connections 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.170+0000 7fbed62f5700 1 -- 172.21.15.37:0/567067290 wait complete. 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 Processor -- start 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 -- start start 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed006b1f0 0x7fbed01019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed0108a60 0x7fbed0102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0112c60 con 0x7fbed006bbb0 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0112ae0 con 0x7fbed006b1f0 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed62f5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0112de0 con 0x7fbed0108a60 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60470/0 (socket says 172.21.15.37:60470) 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 -- 172.21.15.37:0/1275887318 learned_addr learned my addr 172.21.15.37:0/1275887318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecffff700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed006b1f0 0x7fbed01019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbed4892700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed0108a60 0x7fbed0102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 -- 172.21.15.37:0/1275887318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed0108a60 msgr2=0x7fbed0102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed0108a60 0x7fbed0102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 -- 172.21.15.37:0/1275887318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed006b1f0 msgr2=0x7fbed01019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed006b1f0 0x7fbed01019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.171+0000 7fbecf7fe700 1 -- 172.21.15.37:0/1275887318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01a7640 con 0x7fbed006bbb0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbecf7fe700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000c00 tx=0x7fbeb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8027070 con 0x7fbed006bbb0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a78d0 con 0x7fbed006bbb0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb8003fe0 con 0x7fbed006bbb0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8022e30 con 0x7fbed006bbb0 2024-04-25T21:07:14.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.172+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a7d30 con 0x7fbed006bbb0 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.173+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0069730 con 0x7fbed006bbb0 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.174+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbeb802a070 con 0x7fbed006bbb0 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.175+0000 7fbecd7fa700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 0x7fbebc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.175+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbeb80d1380 con 0x7fbed006bbb0 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.175+0000 7fbecffff700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 0x7fbebc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.175+0000 7fbecffff700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 0x7fbebc0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbec000abd0 tx=0x7fbec000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.180+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeb8013070 con 0x7fbed006bbb0 2024-04-25T21:07:14.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.434+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbed002df80 con 0x7fbed006bbb0 2024-04-25T21:07:14.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.435+0000 7fbecd7fa700 1 -- 172.21.15.37:0/1275887318 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7fbeb809dda0 con 0x7fbed006bbb0 2024-04-25T21:07:14.436 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:14.436 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 msgr2=0x7fbebc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 0x7fbebc0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbec000abd0 tx=0x7fbec000a380 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 msgr2=0x7fbed0101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8000c00 tx=0x7fbeb80047b0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 shutdown_connections 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbed0108a60 0x7fbed0102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbebc0a3c40 0x7fbebc0a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbed006b1f0 0x7fbed01019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.437+0000 7fbed62f5700 1 --2- 172.21.15.37:0/1275887318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbed006bbb0 0x7fbed0101ef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:14.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.438+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 >> 172.21.15.37:0/1275887318 conn(0x7fbed00789e0 msgr2=0x7fbed00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.438+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 shutdown_connections 2024-04-25T21:07:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:14.438+0000 7fbed62f5700 1 -- 172.21.15.37:0/1275887318 wait complete. 2024-04-25T21:07:14.439 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:07:15.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:07:15.408 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:15.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:15 smithi037 ceph-mon[29195]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:15.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:15 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1275887318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:15.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:15 smithi177 ceph-mon[33043]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:15.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:15 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1275887318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:15.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:15 smithi098 ceph-mon[32569]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T21:07:15.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:15 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1275887318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 -- 172.21.15.37:0/1041087316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8102300 msgr2=0x7f99e810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 --2- 172.21.15.37:0/1041087316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8102300 0x7f99e810e7f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99e000b0a0 tx=0x7f99e000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 -- 172.21.15.37:0/1041087316 shutdown_connections 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 --2- 172.21.15.37:0/1041087316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8102300 0x7f99e810e7f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 --2- 172.21.15.37:0/1041087316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8107990 0x7f99e8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 --2- 172.21.15.37:0/1041087316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8101940 0x7f99e8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 -- 172.21.15.37:0/1041087316 >> 172.21.15.37:0/1041087316 conn(0x7f99e80788f0 msgr2=0x7f99e8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:17.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 -- 172.21.15.37:0/1041087316 shutdown_connections 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.123+0000 7f99f00a9700 1 -- 172.21.15.37:0/1041087316 wait complete. 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 Processor -- start 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 -- start start 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8101940 0x7f99e819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 0x7f99e819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8111ae0 con 0x7f99e8107990 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8111960 con 0x7f99e8102300 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99f00a9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8111c60 con 0x7f99e8101940 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99ee646700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99ee646700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60474/0 (socket says 172.21.15.37:60474) 2024-04-25T21:07:17.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99ee646700 1 -- 172.21.15.37:0/2755842465 learned_addr learned my addr 172.21.15.37:0/2755842465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99ede45700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8101940 0x7f99e819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.124+0000 7f99ed644700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 0x7f99e819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 -- 172.21.15.37:0/2755842465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8101940 msgr2=0x7f99e819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8101940 0x7f99e819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 -- 172.21.15.37:0/2755842465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 msgr2=0x7f99e819b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 0x7f99e819b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 -- 172.21.15.37:0/2755842465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e81a01b0 con 0x7f99e8107990 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ed644700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 0x7f99e819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99ee646700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f99e00062a0 tx=0x7f99e0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0021070 con 0x7f99e8107990 2024-04-25T21:07:17.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e81a0440 con 0x7f99e8107990 2024-04-25T21:07:17.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e0007c80 con 0x7f99e8107990 2024-04-25T21:07:17.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e001d450 con 0x7f99e8107990 2024-04-25T21:07:17.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.125+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e81a7d20 con 0x7f99e8107990 2024-04-25T21:07:17.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.126+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e8069730 con 0x7f99e8107990 2024-04-25T21:07:17.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.127+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f99e0004000 con 0x7f99e8107990 2024-04-25T21:07:17.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.128+0000 7f99deffd700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 0x7f99d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:17.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.128+0000 7f99ede45700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 0x7f99d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:17.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.128+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f99e000f020 con 0x7f99e8107990 2024-04-25T21:07:17.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.128+0000 7f99ede45700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 0x7f99d40a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99e400abd0 tx=0x7f99e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:17.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.132+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99e002a070 con 0x7f99e8107990 2024-04-25T21:07:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.345+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e81a7f50 con 0x7f99e8107990 2024-04-25T21:07:17.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.346+0000 7f99deffd700 1 -- 172.21.15.37:0/2755842465 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22571 (secure 0 0 0) 0x7f99e009e710 con 0x7f99e8107990 2024-04-25T21:07:17.347 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:17.347 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:35.806342+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 msgr2=0x7f99d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 0x7f99d40a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99e400abd0 tx=0x7f99e400a380 comp rx=0 tx=0).stop 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 msgr2=0x7f99e819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f99e00062a0 tx=0x7f99e0004e20 comp rx=0 tx=0).stop 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 shutdown_connections 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99e8107990 0x7f99e819fa70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99e8101940 0x7f99e819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f99d40a3c40 0x7f99d40a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 --2- 172.21.15.37:0/2755842465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8102300 0x7f99e819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.348+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 >> 172.21.15.37:0/2755842465 conn(0x7f99e80788f0 msgr2=0x7f99e806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.349+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 shutdown_connections 2024-04-25T21:07:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:17.349+0000 7f99f00a9700 1 -- 172.21.15.37:0/2755842465 wait complete. 2024-04-25T21:07:17.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:17 smithi177 ceph-mon[33043]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:17 smithi098 ceph-mon[32569]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:17.722 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:17 smithi037 ceph-mon[29195]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T21:07:18.053 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:07:18.302 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:18.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:18 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2755842465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:18.598 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:18 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2755842465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:18.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:18 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2755842465' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:19 smithi177 ceph-mon[33043]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:19.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:19 smithi098 ceph-mon[32569]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:19.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:19 smithi037 ceph-mon[29195]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 -- 172.21.15.37:0/1342959458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055006b070 msgr2=0x7f055010efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 --2- 172.21.15.37:0/1342959458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055006b070 0x7f055010efb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0544009990 tx=0x7f0544009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 -- 172.21.15.37:0/1342959458 shutdown_connections 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 --2- 172.21.15.37:0/1342959458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055010f4f0 0x7f05501118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 --2- 172.21.15.37:0/1342959458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006a750 0x7f055006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 --2- 172.21.15.37:0/1342959458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055006b070 0x7f055010efb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 -- 172.21.15.37:0/1342959458 >> 172.21.15.37:0/1342959458 conn(0x7f05500788f0 msgr2=0x7f0550078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 -- 172.21.15.37:0/1342959458 shutdown_connections 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.118+0000 7f0556549700 1 -- 172.21.15.37:0/1342959458 wait complete. 2024-04-25T21:07:20.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 Processor -- start 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 -- start start 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055006a750 0x7f055019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 0x7f055019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0550114ef0 con 0x7f055010f4f0 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0550114d70 con 0x7f055006a750 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0556549700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0550115070 con 0x7f055006b070 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0554ae6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0554ae6700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37312/0 (socket says 172.21.15.37:37312) 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f0554ae6700 1 -- 172.21.15.37:0/3056567732 learned_addr learned my addr 172.21.15.37:0/3056567732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f054ffff700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055006a750 0x7f055019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:20.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.119+0000 7f054f7fe700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 0x7f055019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 -- 172.21.15.37:0/3056567732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 msgr2=0x7f055019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 0x7f055019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 -- 172.21.15.37:0/3056567732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055006a750 msgr2=0x7f055019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055006a750 0x7f055019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 -- 172.21.15.37:0/3056567732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05501a0410 con 0x7f055010f4f0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f054f7fe700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 0x7f055019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0554ae6700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f054800b9e0 tx=0x7f054800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548019070 con 0x7f055010f4f0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05501a0700 con 0x7f055010f4f0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f054800cde0 con 0x7f055010f4f0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548012340 con 0x7f055010f4f0 2024-04-25T21:07:20.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.120+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05501a7f80 con 0x7f055010f4f0 2024-04-25T21:07:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.121+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0550069730 con 0x7f055010f4f0 2024-04-25T21:07:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.122+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f054801c070 con 0x7f055010f4f0 2024-04-25T21:07:20.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.123+0000 7f054d7fa700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 0x7f05380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.123+0000 7f054ffff700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 0x7f05380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.123+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f05480cf860 con 0x7f055010f4f0 2024-04-25T21:07:20.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.123+0000 7f054ffff700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 0x7f05380a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0540009a10 tx=0x7f0540009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:20.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.127+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f054809c250 con 0x7f055010f4f0 2024-04-25T21:07:20.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.383+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f055019c760 con 0x7f055010f4f0 2024-04-25T21:07:20.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.384+0000 7f054d7fa700 1 -- 172.21.15.37:0/3056567732 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f054809bb50 con 0x7f055010f4f0 2024-04-25T21:07:20.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:20.385 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 msgr2=0x7f05380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 0x7f05380a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0540009a10 tx=0x7f0540009190 comp rx=0 tx=0).stop 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 msgr2=0x7f055019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f054800b9e0 tx=0x7f054800bcf0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 shutdown_connections 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f055010f4f0 0x7f055019fcd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f05380a3c40 0x7f05380a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f055006a750 0x7f055019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 --2- 172.21.15.37:0/3056567732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f055006b070 0x7f055019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.387+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 >> 172.21.15.37:0/3056567732 conn(0x7f05500788f0 msgr2=0x7f0550110a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.388+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 shutdown_connections 2024-04-25T21:07:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:20.388+0000 7f0556549700 1 -- 172.21.15.37:0/3056567732 wait complete. 2024-04-25T21:07:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:07:21.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T21:05:03.820512+0000', 'modified': '2024-04-25T21:05:22.903199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24445, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6839/2484283883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 2484283883}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 2484283883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14574': {'gid': 14574, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6841/2041063205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 2041063205}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 2041063205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24380': {'gid': 24380, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6833/3425764534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 3425764534}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 3425764534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/785792135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 785792135}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 785792135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6833/3715259407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3715259407}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 3715259407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.98:6837/2446670971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2446670971}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2446670971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.177:6835/724905952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 724905952}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 724905952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.177:6837/2828024557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 2828024557}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 2828024557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.98:6839/899178427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 899178427}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 899178427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6839/3063347892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3063347892}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3063347892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24458, 24469, 24457, 24445, 24419]} 2024-04-25T21:07:21.119 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T21:07:21.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T21:07:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:21.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:21 smithi037 ceph-mon[29195]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:21.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3056567732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:21 smithi177 ceph-mon[33043]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:21.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3056567732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:21 smithi098 ceph-mon[32569]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3056567732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- 172.21.15.37:0/3258639994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4102ac0 msgr2=0x7faed4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 --2- 172.21.15.37:0/3258639994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4102ac0 0x7faed4102f40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faec8009990 tx=0x7faec8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- 172.21.15.37:0/3258639994 shutdown_connections 2024-04-25T21:07:23.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 --2- 172.21.15.37:0/3258639994 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4103480 0x7faed410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 --2- 172.21.15.37:0/3258639994 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 --2- 172.21.15.37:0/3258639994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4102ac0 0x7faed4102f40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- 172.21.15.37:0/3258639994 >> 172.21.15.37:0/3258639994 conn(0x7faed4079bb0 msgr2=0x7faed4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- 172.21.15.37:0/3258639994 shutdown_connections 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- 172.21.15.37:0/3258639994 wait complete. 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 Processor -- start 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.163+0000 7faedc2f5700 1 -- start start 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 0x7faed41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faed4112ca0 con 0x7faed4103480 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faed4112b20 con 0x7faed4108a70 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faedc2f5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faed4112e20 con 0x7faed4102ac0 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37336/0 (socket says 172.21.15.37:37336) 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 -- 172.21.15.37:0/536680459 learned_addr learned my addr 172.21.15.37:0/536680459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faeda892700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faeda091700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 0x7faed41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 -- 172.21.15.37:0/536680459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 msgr2=0x7faed41990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 0x7faed41990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 -- 172.21.15.37:0/536680459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 msgr2=0x7faed419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 -- 172.21.15.37:0/536680459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed419e100 con 0x7faed4103480 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faeda091700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 0x7faed41990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faeda892700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed9890700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.164+0000 7faed37fe700 1 -- 172.21.15.37:0/536680459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8027070 con 0x7faed4103480 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.165+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed419e390 con 0x7faed4103480 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.165+0000 7faed37fe700 1 -- 172.21.15.37:0/536680459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec80041d0 con 0x7faed4103480 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.165+0000 7faed37fe700 1 -- 172.21.15.37:0/536680459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800f460 con 0x7faed4103480 2024-04-25T21:07:23.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.165+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed41a5c70 con 0x7faed4103480 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.167+0000 7faed37fe700 1 -- 172.21.15.37:0/536680459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faec802a070 con 0x7faed4103480 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.167+0000 7faed37fe700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 0x7faebc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.167+0000 7faeda091700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 0x7faebc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.167+0000 7faed37fe700 1 -- 172.21.15.37:0/536680459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7faec80d13d0 con 0x7faed4103480 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.168+0000 7faeda091700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 0x7faebc0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faec400e300 tx=0x7faec4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.168+0000 7faee6b7eb80 1 Processor -- start 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.168+0000 7faee6b7eb80 1 -- start start 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b248f1ce0 con 0x557b249a4500 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b24898910 con 0x557b249a3be0 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557b248edca0 con 0x557b249a8990 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37344/0 (socket says 172.21.15.37:37344) 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 -- 172.21.15.37:0/2040924188 learned_addr learned my addr 172.21.15.37:0/2040924188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faeda091700 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 -- 172.21.15.37:0/2040924188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 msgr2=0x557b249a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 -- 172.21.15.37:0/2040924188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 msgr2=0x557b249a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 -- 172.21.15.37:0/2040924188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b2483f880 con 0x557b249a4500 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed9890700 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faec809a120 tx=0x7faec809d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac66bf746c233c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed0d70700 1 -- 172.21.15.37:0/2040924188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8022470 con 0x557b249a4500 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed0d70700 1 -- 172.21.15.37:0/2040924188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec809d750 con 0x557b249a4500 2024-04-25T21:07:23.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.169+0000 7faed0d70700 1 -- 172.21.15.37:0/2040924188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec809dc10 con 0x557b249a4500 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2040924188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 msgr2=0x557b249a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7faec809a120 tx=0x7faec809d360 comp rx=0 tx=0).stop 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2040924188 shutdown_connections 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2040924188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249a8450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2040924188 >> 172.21.15.37:0/2040924188 conn(0x557b248cc640 msgr2=0x557b2499a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2040924188 shutdown_connections 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.170+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2040924188 wait complete. 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 -- start start 2024-04-25T21:07:23.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 Processor -- start 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a3be0 0x557b249aff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 0x557b249ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b248f1ce0 con 0x557b249a4500 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b24890b80 con 0x557b249a8990 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557b249b0870 con 0x557b249a3be0 2024-04-25T21:07:23.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faee6b7eb80 10 client.0 _ll_register_callbacks cb 0x557b2488a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37354/0 (socket says 172.21.15.37:37354) 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 -- 172.21.15.37:0/2920876935 learned_addr learned my addr 172.21.15.37:0/2920876935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faeda892700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 0x557b249ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faeda091700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a3be0 0x557b249aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 -- 172.21.15.37:0/2920876935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a3be0 msgr2=0x557b249aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a3be0 0x557b249aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 -- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 msgr2=0x557b249ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 0x557b249ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faed9890700 1 -- 172.21.15.37:0/2920876935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b248a9220 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.171+0000 7faeda892700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 0x557b249ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faed9890700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faec80cef30 tx=0x7faec8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2ab53853171c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80b5070 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec809d750 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7faec809b7f0 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faec8022470 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 10 client.14931 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b249bb000 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec809d750 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faec8086ce0 con 0x557b249a4500 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 10 client.14931 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 10 client.14931 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:07:23.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 20 client.14931 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b249bd370 0x557b249be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 4 client.14931 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.172+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b248ff4d0 con 0x557b249bd370 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.173+0000 7faeda091700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b249bd370 0x557b249be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.173+0000 7faeda091700 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b249bd370 0x557b249be7e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e97a29448071bced server_cookie=36e363fe39e7206 in_seq=0 out_seq=0 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.173+0000 7faeb9ffb700 10 client.14931 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faeb9ffb700 1 -- 172.21.15.37:0/2920876935 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b248ff4d0 con 0x557b249bd370 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faeb9ffb700 10 client.14931 handle_command_reply: tid=0 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 client.14931 shutdown 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 msgr2=0x557b249ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faec80cef30 tx=0x7faec8005c00 comp rx=0 tx=0).stop 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 shutdown_connections 2024-04-25T21:07:23.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a8990 0x557b249ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a3be0 0x557b249aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b249bd370 0x557b249be7e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2920876935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a4500 0x557b249ad210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 >> 172.21.15.37:0/2920876935 conn(0x557b248cc640 msgr2=0x557b249a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faeb9ffb700 0 client.14931 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 shutdown_connections 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2920876935 wait complete. 2024-04-25T21:07:23.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.174+0000 7faee6b7eb80 20 client.14931 trim_cache size 0 max 16384 2024-04-25T21:07:23.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 Processor -- start 2024-04-25T21:07:23.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 -- start start 2024-04-25T21:07:23.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a4500 0x557b249aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b249bcb20 con 0x557b249a3be0 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b249bd1f0 con 0x557b249a4500 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557b249bd8c0 con 0x557b249a8990 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.195+0000 7faeda091700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37360/0 (socket says 172.21.15.37:37360) 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 -- 172.21.15.37:0/2209941476 learned_addr learned my addr 172.21.15.37:0/2209941476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 -- 172.21.15.37:0/2209941476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 msgr2=0x557b249aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 -- 172.21.15.37:0/2209941476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a4500 msgr2=0x557b249aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faed9890700 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a4500 0x557b249aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a4500 0x557b249aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 -- 172.21.15.37:0/2209941476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b2499a120 con 0x557b249a3be0 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faeda091700 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faec4007b40 tx=0x7faec4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5a6beff76068b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faebbfff700 1 -- 172.21.15.37:0/2209941476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4015da0 con 0x557b249a3be0 2024-04-25T21:07:23.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faebbfff700 1 -- 172.21.15.37:0/2209941476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec4028150 con 0x557b249a3be0 2024-04-25T21:07:23.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.196+0000 7faebbfff700 1 -- 172.21.15.37:0/2209941476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4004020 con 0x557b249a3be0 2024-04-25T21:07:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2209941476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 msgr2=0x557b249ac210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faec4007b40 tx=0x7faec4014000 comp rx=0 tx=0).stop 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2209941476 shutdown_connections 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a8990 0x557b249aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a3be0 0x557b249ac210 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/2209941476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a4500 0x557b249aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2209941476 >> 172.21.15.37:0/2209941476 conn(0x557b248cc640 msgr2=0x557b249abb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2209941476 shutdown_connections 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- 172.21.15.37:0/2209941476 wait complete. 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- start start 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 Processor -- start 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a4500 0x557b249c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557b249be660 con 0x557b249a8990 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557b249bdf90 con 0x557b249a3be0 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.197+0000 7faee6b7eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557b249bed30 con 0x557b249a4500 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faee6b7eb80 10 client.0 _ll_register_callbacks cb 0x557b2488a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37364/0 (socket says 172.21.15.37:37364) 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 -- 172.21.15.37:0/3733038094 learned_addr learned my addr 172.21.15.37:0/3733038094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda091700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faed9890700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a4500 0x557b249c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 -- 172.21.15.37:0/3733038094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a4500 msgr2=0x557b249c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a4500 0x557b249c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 -- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 msgr2=0x557b249c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 -- 172.21.15.37:0/3733038094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b249c1440 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeda892700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faecc00bdd0 tx=0x7faecc00f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5159c225724ca962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc018020 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec4028150 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7faecc014460 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faecc014660 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faee6b7eb80 10 client.14937 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.198+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b24a24fe0 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec4028150 con 0x557b249a8990 2024-04-25T21:07:23.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faecc013790 con 0x557b249a8990 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 10 client.14937 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 10 client.14937 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 20 client.14937 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b24a29be0 0x557b24a288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 4 client.14937 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "status"}) -- 0x557b248f9570 con 0x557b24a29be0 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.199+0000 7faeda091700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b24a29be0 0x557b24a288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:23.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faeda091700 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b24a29be0 0x557b24a288f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa7522bcd6a694ad server_cookie=7f36911fb20b1dfc in_seq=0 out_seq=0 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faeb9ffb700 10 client.14937 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faeb9ffb700 1 -- 172.21.15.37:0/3733038094 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557b248f9570 con 0x557b24a29be0 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faeb9ffb700 10 client.14937 handle_command_reply: tid=0 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 client.14937 shutdown 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 msgr2=0x557b249c0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faecc00bdd0 tx=0x7faecc00f4c0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 shutdown_connections 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x557b24a29be0 0x557b24a288f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557b249a8990 0x557b249c0c00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557b249a3be0 0x557b249c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faeb9ffb700 0 client.14937 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:07:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 --2- 172.21.15.37:0/3733038094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557b249a4500 0x557b249c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 >> 172.21.15.37:0/3733038094 conn(0x557b248cc640 msgr2=0x557b2499e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 shutdown_connections 2024-04-25T21:07:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 1 -- 172.21.15.37:0/3733038094 wait complete. 2024-04-25T21:07:23.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.200+0000 7faee6b7eb80 20 client.14937 trim_cache size 0 max 16384 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 4, 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24419, 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:07:23.208 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 123.43495796400001, 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 170.14580565700001 2024-04-25T21:07:23.209 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 msgr2=0x7faebc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 0x7faebc0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faec400e300 tx=0x7faec4009040 comp rx=0 tx=0).stop 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 msgr2=0x7faed4199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7faec8000c00 tx=0x7faec80047b0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 shutdown_connections 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7faebc0a3c40 0x7faebc0a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faed4108a70 0x7faed419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faed4102ac0 0x7faed41990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 --2- 172.21.15.37:0/536680459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faed4103480 0x7faed4199630 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:23.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 >> 172.21.15.37:0/536680459 conn(0x7faed4079bb0 msgr2=0x7faed41016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:23.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 shutdown_connections 2024-04-25T21:07:23.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:23.201+0000 7faedc2f5700 1 -- 172.21.15.37:0/536680459 wait complete. 2024-04-25T21:07:23.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:23 smithi177 ceph-mon[33043]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:23.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:23 smithi098 ceph-mon[32569]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:23.693 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:23 smithi037 ceph-mon[29195]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:24.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.f status 2024-04-25T21:07:24.273 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:25.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:25 smithi037 ceph-mon[29195]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:25 smithi177 ceph-mon[33043]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:25.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:25 smithi098 ceph-mon[32569]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:25.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 -- 172.21.15.37:0/1697213600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 msgr2=0x7f1708107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 --2- 172.21.15.37:0/1697213600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f1708107d70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009990 tx=0x7f16f8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 -- 172.21.15.37:0/1697213600 shutdown_connections 2024-04-25T21:07:25.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 --2- 172.21.15.37:0/1697213600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708102300 0x7f170810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 --2- 172.21.15.37:0/1697213600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f1708107d70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 --2- 172.21.15.37:0/1697213600 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708101940 0x7f1708101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 -- 172.21.15.37:0/1697213600 >> 172.21.15.37:0/1697213600 conn(0x7f17080788f0 msgr2=0x7f1708078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 -- 172.21.15.37:0/1697213600 shutdown_connections 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.943+0000 7f170e71f700 1 -- 172.21.15.37:0/1697213600 wait complete. 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 Processor -- start 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 -- start start 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 0x7f170819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 0x7f170819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111bd0 con 0x7f1708107990 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111a50 con 0x7f1708102300 2024-04-25T21:07:25.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170e71f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1708111d50 con 0x7f1708101940 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37370/0 (socket says 172.21.15.37:37370) 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 -- 172.21.15.37:0/1659049219 learned_addr learned my addr 172.21.15.37:0/1659049219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f1707fff700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 0x7f170819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 -- 172.21.15.37:0/1659049219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 msgr2=0x7f170819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 0x7f170819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f170ccbc700 1 -- 172.21.15.37:0/1659049219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 msgr2=0x7f170819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.944+0000 7f17077fe700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 0x7f170819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f170ccbc700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 0x7f170819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f170ccbc700 1 -- 172.21.15.37:0/1659049219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17081a5690 con 0x7f1708107990 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f1707fff700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 0x7f170819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f17077fe700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 0x7f170819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f170ccbc700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f170000b970 tx=0x7f170000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f17057fa700 1 -- 172.21.15.37:0/1659049219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700019070 con 0x7f1708107990 2024-04-25T21:07:25.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f17057fa700 1 -- 172.21.15.37:0/1659049219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1700014440 con 0x7f1708107990 2024-04-25T21:07:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17081a58c0 con 0x7f1708107990 2024-04-25T21:07:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f17057fa700 1 -- 172.21.15.37:0/1659049219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170000ce60 con 0x7f1708107990 2024-04-25T21:07:25.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.945+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17081a5e20 con 0x7f1708107990 2024-04-25T21:07:25.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.947+0000 7f17057fa700 1 -- 172.21.15.37:0/1659049219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f170001c070 con 0x7f1708107990 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f17057fa700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 0x7f16f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f1707fff700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 0x7f16f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f17057fa700 1 -- 172.21.15.37:0/1659049219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f17000cf740 con 0x7f1708107990 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f1707fff700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 0x7f16f40a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16f8006010 tx=0x7f16f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f1718fa8b80 1 Processor -- start 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.948+0000 7f1718fa8b80 1 -- start start 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b500d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5018340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c3b4f60c40 con 0x55c3b5018880 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c3b4f078e0 con 0x55c3b500d9d0 2024-04-25T21:07:25.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3b4f5cc00 con 0x55c3b500d0b0 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37382/0 (socket says 172.21.15.37:37382) 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 -- 172.21.15.37:0/2668719437 learned_addr learned my addr 172.21.15.37:0/2668719437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f17077fe700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5018340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1707fff700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b500d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 -- 172.21.15.37:0/2668719437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 msgr2=0x55c3b500d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b500d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 -- 172.21.15.37:0/2668719437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 msgr2=0x55c3b5018340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5018340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 -- 172.21.15.37:0/2668719437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b4ecb6f0 con 0x55c3b5018880 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f1707fff700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b500d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f170ccbc700 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17000a4170 tx=0x7f1700004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47d956e4fca873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.949+0000 7f16fe7fc700 1 -- 172.21.15.37:0/2668719437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000bc0b0 con 0x55c3b5018880 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f16fe7fc700 1 -- 172.21.15.37:0/2668719437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f170009fde0 con 0x55c3b5018880 2024-04-25T21:07:25.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f16fe7fc700 1 -- 172.21.15.37:0/2668719437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17000a4210 con 0x55c3b5018880 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 -- 172.21.15.37:0/2668719437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 msgr2=0x55c3b501ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f17000a4170 tx=0x7f1700004be0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 -- 172.21.15.37:0/2668719437 shutdown_connections 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b5018880 0x55c3b501ac70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b500d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/2668719437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5018340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 -- 172.21.15.37:0/2668719437 >> 172.21.15.37:0/2668719437 conn(0x55c3b4f3a840 msgr2=0x55c3b5009580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 -- 172.21.15.37:0/2668719437 shutdown_connections 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.950+0000 7f1718fa8b80 1 -- 172.21.15.37:0/2668719437 wait complete. 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 -- start start 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 Processor -- start 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b5017fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 0x55c3b5015800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c3b501b370 con 0x55c3b500d9d0 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c3b4f1a560 con 0x55c3b5018880 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3b501b4f0 con 0x55c3b500d0b0 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1718fa8b80 10 client.0 _ll_register_callbacks cb 0x55c3b4ef98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f17077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f1707fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b5017fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f17077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37398/0 (socket says 172.21.15.37:37398) 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.951+0000 7f17077fe700 1 -- 172.21.15.37:0/1665838963 learned_addr learned my addr 172.21.15.37:0/1665838963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f170ccbc700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 0x55c3b5015800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 -- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 msgr2=0x55c3b5017fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b5017fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 -- 172.21.15.37:0/1665838963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 msgr2=0x55c3b5015800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 0x55c3b5015800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 -- 172.21.15.37:0/1665838963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b4eae880 con 0x55c3b500d9d0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f170ccbc700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 0x55c3b5015800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f17077fe700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f16f000df60 tx=0x7f16f000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc065c3cdb27fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f16e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0018070 con 0x55c3b500d9d0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f16f0014480 con 0x55c3b500d9d0 2024-04-25T21:07:25.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f170009fde0 con 0x55c3b500d9d0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16f0024050 con 0x55c3b500d9d0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f1718fa8b80 10 client.14949 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3b5025bb0 con 0x55c3b500d9d0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.952+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f170009fde0 con 0x55c3b500d9d0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16f00137d0 con 0x55c3b500d9d0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 10 client.14949 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 10 client.14949 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 20 client.14949 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5029650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 4 client.14949 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3b4f6e430 con 0x55c3b50281e0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1707fff700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5029650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f1707fff700 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5029650 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a92957608dac5e1a server_cookie=1a75f9ad84de5172 in_seq=0 out_seq=0 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.953+0000 7f16e77fe700 10 client.14949 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.954+0000 7f16e77fe700 1 -- 172.21.15.37:0/1665838963 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3b4f6e430 con 0x55c3b50281e0 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.954+0000 7f16e77fe700 10 client.14949 handle_command_reply: tid=0 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.954+0000 7f1718fa8b80 1 client.14949 shutdown 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 msgr2=0x55c3b50152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f16f000df60 tx=0x7f16f000df90 comp rx=0 tx=0).stop 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 shutdown_connections 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5029650 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b5018880 0x55c3b5015800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b500d0b0 0x55c3b5017fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f16e77fe700 0 client.14949 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/1665838963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b50152c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 >> 172.21.15.37:0/1665838963 conn(0x55c3b4f3a840 msgr2=0x55c3b50091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 shutdown_connections 2024-04-25T21:07:25.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 1 -- 172.21.15.37:0/1665838963 wait complete. 2024-04-25T21:07:25.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.955+0000 7f1718fa8b80 20 client.14949 trim_cache size 0 max 16384 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 Processor -- start 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 -- start start 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 0x55c3b5032ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b50095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c3b5027990 con 0x55c3b500d9d0 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c3b5028060 con 0x55c3b500d0b0 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3b5028730 con 0x55c3b5018880 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37412/0 (socket says 172.21.15.37:37412) 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 -- 172.21.15.37:0/4216348801 learned_addr learned my addr 172.21.15.37:0/4216348801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1707fff700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 0x55c3b5032ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f170ccbc700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b50095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 -- 172.21.15.37:0/4216348801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 msgr2=0x55c3b50095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b50095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 -- 172.21.15.37:0/4216348801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 msgr2=0x55c3b5032ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 0x55c3b5032ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 -- 172.21.15.37:0/4216348801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b500a2a0 con 0x55c3b500d9d0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f1707fff700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 0x55c3b5032ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f17077fe700 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f16f00048a0 tx=0x7f16f000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a5e485480710d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.976+0000 7f16fdffb700 1 -- 172.21.15.37:0/4216348801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0018040 con 0x55c3b500d9d0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f16fdffb700 1 -- 172.21.15.37:0/4216348801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f000fd90 con 0x55c3b500d9d0 2024-04-25T21:07:25.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f16fdffb700 1 -- 172.21.15.37:0/4216348801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f001dc20 con 0x55c3b500d9d0 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 -- 172.21.15.37:0/4216348801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 msgr2=0x55c3b5009060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f16f00048a0 tx=0x7f16f000f850 comp rx=0 tx=0).stop 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 -- 172.21.15.37:0/4216348801 shutdown_connections 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b50095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d0b0 0x55c3b5032ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/4216348801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d9d0 0x55c3b5009060 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 -- 172.21.15.37:0/4216348801 >> 172.21.15.37:0/4216348801 conn(0x55c3b4f3a840 msgr2=0x55c3b5017c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 -- 172.21.15.37:0/4216348801 shutdown_connections 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.977+0000 7f1718fa8b80 1 -- 172.21.15.37:0/4216348801 wait complete. 2024-04-25T21:07:25.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 -- start start 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 Processor -- start 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5012da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b5011590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c3b50294d0 con 0x55c3b500d0b0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c3b5028e00 con 0x55c3b500d9d0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c3b5029ba0 con 0x55c3b5018880 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1718fa8b80 10 client.0 _ll_register_callbacks cb 0x55c3b4ef98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1707fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1707fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37428/0 (socket says 172.21.15.37:37428) 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f1707fff700 1 -- 172.21.15.37:0/323424434 learned_addr learned my addr 172.21.15.37:0/323424434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f17077fe700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5012da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.978+0000 7f170ccbc700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b5011590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 -- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 msgr2=0x55c3b5011590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b5011590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 -- 172.21.15.37:0/323424434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 msgr2=0x55c3b5012da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5012da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 -- 172.21.15.37:0/323424434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b5011d40 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f17077fe700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5012da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1707fff700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009990 tx=0x7f16f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd43618d7072089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f800fe60 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f16f8034330 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f000fd90 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16f800be60 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1718fa8b80 10 client.14955 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3b5093dd0 con 0x55c3b500d0b0 2024-04-25T21:07:25.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f000fd90 con 0x55c3b500d0b0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f16f803dae0 con 0x55c3b500d0b0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1718fa8b80 10 client.14955 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.979+0000 7f1718fa8b80 10 client.14955 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f1718fa8b80 20 client.14955 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5097720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f1718fa8b80 4 client.14955 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "status"}) -- 0x55c3b4f684d0 con 0x55c3b50281e0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f17077fe700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5097720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f16e77fe700 10 client.14955 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f17077fe700 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5097720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5be1c28215cc6612 server_cookie=95cb2014be246a44 in_seq=0 out_seq=0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f16e77fe700 1 -- 172.21.15.37:0/323424434 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c3b4f684d0 con 0x55c3b50281e0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f16e77fe700 10 client.14955 handle_command_reply: tid=0 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.980+0000 7f1718fa8b80 1 client.14955 shutdown 2024-04-25T21:07:25.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 msgr2=0x55c3b5012860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009990 tx=0x7f16f8009310 comp rx=0 tx=0).stop 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 shutdown_connections 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c3b50281e0 0x55c3b5097720 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c3b5018880 0x55c3b5011590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f16e77fe700 0 client.14955 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3b500d0b0 0x55c3b5012860 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 --2- 172.21.15.37:0/323424434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c3b500d9d0 0x55c3b5012da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 >> 172.21.15.37:0/323424434 conn(0x55c3b4f3a840 msgr2=0x55c3b5010240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 shutdown_connections 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 1 -- 172.21.15.37:0/323424434 wait complete. 2024-04-25T21:07:25.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f1718fa8b80 20 client.14955 trim_cache size 0 max 16384 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 3, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24445, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 127.221335462, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:07:25.987 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 178.75298280199999 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 msgr2=0x7f16f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 0x7f16f40a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f16f8006010 tx=0x7f16f801a040 comp rx=0 tx=0).stop 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 msgr2=0x7f17081a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f170000b970 tx=0x7f170000bd30 comp rx=0 tx=0).stop 2024-04-25T21:07:25.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.981+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 shutdown_connections 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1708107990 0x7f17081a5150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1708101940 0x7f170819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f16f40a3cc0 0x7f16f40a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 --2- 172.21.15.37:0/1659049219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1708102300 0x7f170819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 >> 172.21.15.37:0/1659049219 conn(0x7f17080788f0 msgr2=0x7f1708100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 shutdown_connections 2024-04-25T21:07:25.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:25.982+0000 7f170e71f700 1 -- 172.21.15.37:0/1659049219 wait complete. 2024-04-25T21:07:26.673 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T21:07:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:27.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:27 smithi177 ceph-mon[33043]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:27 smithi098 ceph-mon[32569]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:27.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:27 smithi037 ceph-mon[29195]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 -- 172.21.15.37:0/1084362347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 msgr2=0x7f5ddc06b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 --2- 172.21.15.37:0/1084362347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc06b5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009990 tx=0x7f5dcc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 -- 172.21.15.37:0/1084362347 shutdown_connections 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 --2- 172.21.15.37:0/1084362347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 0x7f5ddc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 --2- 172.21.15.37:0/1084362347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 --2- 172.21.15.37:0/1084362347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc06b5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 -- 172.21.15.37:0/1084362347 >> 172.21.15.37:0/1084362347 conn(0x7f5ddc0788f0 msgr2=0x7f5ddc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 -- 172.21.15.37:0/1084362347 shutdown_connections 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.623+0000 7f5de3977700 1 -- 172.21.15.37:0/1084362347 wait complete. 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 Processor -- start 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 -- start start 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 0x7f5ddc101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc1024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc112c70 con 0x7f5ddc06b170 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc112af0 con 0x7f5ddc108a30 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de3977700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc112df0 con 0x7f5ddc06bb10 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58518/0 (socket says 172.21.15.37:58518) 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 -- 172.21.15.37:0/530466238 learned_addr learned my addr 172.21.15.37:0/530466238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de0f12700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 0x7f5ddc101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1f14700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc1024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 -- 172.21.15.37:0/530466238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 msgr2=0x7f5ddc101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 0x7f5ddc101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 -- 172.21.15.37:0/530466238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 msgr2=0x7f5ddc1024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc1024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1713700 1 -- 172.21.15.37:0/530466238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a7650 con 0x7f5ddc06b170 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.624+0000 7f5de1f14700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5de1713700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c9b0 tx=0x7f5dd800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5dd27fc700 1 -- 172.21.15.37:0/530466238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8007b00 con 0x7f5ddc06b170 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a7940 con 0x7f5ddc06b170 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5dd27fc700 1 -- 172.21.15.37:0/530466238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8007ca0 con 0x7f5ddc06b170 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5dd27fc700 1 -- 172.21.15.37:0/530466238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8018730 con 0x7f5ddc06b170 2024-04-25T21:07:28.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.625+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a7e80 con 0x7f5ddc06b170 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.627+0000 7f5dd27fc700 1 -- 172.21.15.37:0/530466238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5dd800f450 con 0x7f5ddc06b170 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.627+0000 7f5dd27fc700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 0x7f5dc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.628+0000 7f5de0f12700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 0x7f5dc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.628+0000 7f5dd27fc700 1 -- 172.21.15.37:0/530466238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5dd80d0450 con 0x7f5ddc06b170 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.628+0000 7f5de0f12700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 0x7f5dc80a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b5c0 tx=0x7f5dcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 Processor -- start 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 -- start start 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6becff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff9d0 0x55b6bed0a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6bec52c40 con 0x55b6bed0a880 2024-04-25T21:07:28.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6bebf98e0 con 0x55b6becff9d0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5dee200b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6bec4ec00 con 0x55b6becff0b0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58526/0 (socket says 172.21.15.37:58526) 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 -- 172.21.15.37:0/215158400 learned_addr learned my addr 172.21.15.37:0/215158400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1713700 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6becff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de0f12700 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff9d0 0x55b6bed0a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 -- 172.21.15.37:0/215158400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 msgr2=0x55b6becff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6becff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 -- 172.21.15.37:0/215158400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff9d0 msgr2=0x55b6bed0a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff9d0 0x55b6bed0a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.629+0000 7f5de1f14700 1 -- 172.21.15.37:0/215158400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6bebbd6f0 con 0x55b6bed0a880 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5de1f14700 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007f30 tx=0x7f5dd400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a82307228a71b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5dc77fe700 1 -- 172.21.15.37:0/215158400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4014070 con 0x55b6bed0a880 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5dc77fe700 1 -- 172.21.15.37:0/215158400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd400f8f0 con 0x55b6bed0a880 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5dc77fe700 1 -- 172.21.15.37:0/215158400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400e930 con 0x55b6bed0a880 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5dee200b80 1 -- 172.21.15.37:0/215158400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 msgr2=0x55b6bed0cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.630+0000 7f5dee200b80 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007f30 tx=0x7f5dd400c5a0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 -- 172.21.15.37:0/215158400 shutdown_connections 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed0a880 0x55b6bed0cc70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6becff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- 172.21.15.37:0/215158400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff9d0 0x55b6bed0a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 -- 172.21.15.37:0/215158400 >> 172.21.15.37:0/215158400 conn(0x55b6bec2c840 msgr2=0x55b6becfb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 -- 172.21.15.37:0/215158400 shutdown_connections 2024-04-25T21:07:28.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 -- 172.21.15.37:0/215158400 wait complete. 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 -- start start 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 Processor -- start 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6bed09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.631+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6bed0a880 0x55b6bed07780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5dee200b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed0d370 con 0x55b6becff9d0 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5dee200b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6bec0c560 con 0x55b6bed0a880 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5dee200b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed0d4f0 con 0x55b6becff0b0 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5dee200b80 10 client.0 _ll_register_callbacks cb 0x55b6bebeb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58536/0 (socket says 172.21.15.37:58536) 2024-04-25T21:07:28.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 -- 172.21.15.37:0/3433643000 learned_addr learned my addr 172.21.15.37:0/3433643000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de1f14700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6bed0a880 0x55b6bed07780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de1713700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6bed09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 -- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 msgr2=0x55b6bed09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6bed09f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 -- 172.21.15.37:0/3433643000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6bed0a880 msgr2=0x55b6bed07780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6bed0a880 0x55b6bed07780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 -- 172.21.15.37:0/3433643000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6beba0880 con 0x55b6becff9d0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5de0f12700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc015000 tx=0x7f5dcc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7944d9bd4954adea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.632+0000 7f5dc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc003e20 con 0x55b6becff9d0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5dcc01e030 con 0x55b6becff9d0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd400f8f0 con 0x55b6becff9d0 2024-04-25T21:07:28.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dcc00b9e0 con 0x55b6becff9d0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 10 client.14967 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6bed17ba0 con 0x55b6becff9d0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd400f8f0 con 0x55b6becff9d0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dcc00fb20 con 0x55b6becff9d0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 10 client.14967 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 10 client.14967 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 20 client.14967 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed1a210 0x55b6bed1b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 4 client.14967 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.633+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6bec60430 con 0x55b6bed1a210 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.634+0000 7f5de1713700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed1a210 0x55b6bed1b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.634+0000 7f5de1713700 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed1a210 0x55b6bed1b680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7616ec225d77aad9 server_cookie=48a968ccb19a41be in_seq=0 out_seq=0 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.634+0000 7f5dc4ff9700 10 client.14967 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:07:28.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/3433643000 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6bec60430 con 0x55b6bed1a210 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dc4ff9700 10 client.14967 handle_command_reply: tid=0 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 client.14967 shutdown 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 msgr2=0x55b6bed07240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc015000 tx=0x7f5dcc009310 comp rx=0 tx=0).stop 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 shutdown_connections 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6bed0a880 0x55b6bed07780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed1a210 0x55b6bed1b680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6becff0b0 0x55b6bed09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 --2- 172.21.15.37:0/3433643000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6becff9d0 0x55b6bed07240 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.635+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 >> 172.21.15.37:0/3433643000 conn(0x55b6bec2c840 msgr2=0x55b6becfb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.636+0000 7f5dc4ff9700 0 client.14967 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.636+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 shutdown_connections 2024-04-25T21:07:28.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.636+0000 7f5dee200b80 1 -- 172.21.15.37:0/3433643000 wait complete. 2024-04-25T21:07:28.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.636+0000 7f5dee200b80 20 client.14967 trim_cache size 0 max 16384 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 Processor -- start 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 -- start start 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed07200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed07740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed199c0 con 0x55b6bed24ae0 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed1a090 con 0x55b6becff0b0 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.656+0000 7f5dee200b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed1a760 con 0x55b6bed0a880 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58538/0 (socket says 172.21.15.37:58538) 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 -- 172.21.15.37:0/2160501555 learned_addr learned my addr 172.21.15.37:0/2160501555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1713700 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed07200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 -- 172.21.15.37:0/2160501555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 msgr2=0x55b6bed07740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed07740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 -- 172.21.15.37:0/2160501555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 msgr2=0x55b6bed07200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed07200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 -- 172.21.15.37:0/2160501555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6bed00f20 con 0x55b6bed24ae0 2024-04-25T21:07:28.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1713700 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed07200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4004b10 tx=0x7f5dd400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a15aec5a2ead2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5dc6ffd700 1 -- 172.21.15.37:0/2160501555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4028070 con 0x55b6bed24ae0 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5dc6ffd700 1 -- 172.21.15.37:0/2160501555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd400ea90 con 0x55b6bed24ae0 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.657+0000 7f5dc6ffd700 1 -- 172.21.15.37:0/2160501555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4014040 con 0x55b6bed24ae0 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- 172.21.15.37:0/2160501555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 msgr2=0x55b6bed24f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4004b10 tx=0x7f5dd400cf40 comp rx=0 tx=0).stop 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- 172.21.15.37:0/2160501555 shutdown_connections 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed24f20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed07200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2160501555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed07740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- 172.21.15.37:0/2160501555 >> 172.21.15.37:0/2160501555 conn(0x55b6bec2c840 msgr2=0x55b6bed19010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- 172.21.15.37:0/2160501555 shutdown_connections 2024-04-25T21:07:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- 172.21.15.37:0/2160501555 wait complete. 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 -- start start 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.658+0000 7f5dee200b80 1 Processor -- start 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed1dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed1b500 con 0x55b6bed24ae0 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed1ae30 con 0x55b6becff0b0 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b6bed1bbd0 con 0x55b6bed0a880 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dee200b80 10 client.0 _ll_register_callbacks cb 0x55b6bebeb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58550/0 (socket says 172.21.15.37:58550) 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 -- 172.21.15.37:0/2174524936 learned_addr learned my addr 172.21.15.37:0/2174524936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:28.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1713700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de0f12700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed1dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 -- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 msgr2=0x55b6bed1dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed1dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 -- 172.21.15.37:0/2174524936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 msgr2=0x55b6bed1daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed1daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 -- 172.21.15.37:0/2174524936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6bed1c5e0 con 0x55b6bed24ae0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5de1f14700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007f00 tx=0x7f5dd40077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be336c9ccb748cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.659+0000 7f5dc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4014030 con 0x55b6bed24ae0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd400ea90 con 0x55b6bed24ae0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5dd40236c0 con 0x55b6bed24ae0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dd4028070 con 0x55b6bed24ae0 2024-04-25T21:07:28.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 10 client.14973 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b6becfc2a0 con 0x55b6bed24ae0 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd400ea90 con 0x55b6bed24ae0 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dd4034030 con 0x55b6bed24ae0 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 10 client.14973 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 10 client.14973 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 20 client.14973 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed86790 0x55b6bed854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 4 client.14973 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.660+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "status"}) -- 0x55b6bec5a4d0 con 0x55b6bed86790 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5de1713700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed86790 0x55b6bed854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dc4ff9700 10 client.14973 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5de1713700 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed86790 0x55b6bed854a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5b955755e509c0f server_cookie=7c8967f4cfec33cf in_seq=0 out_seq=0 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dc4ff9700 1 -- 172.21.15.37:0/2174524936 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b6bec5a4d0 con 0x55b6bed86790 2024-04-25T21:07:28.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dc4ff9700 10 client.14973 handle_command_reply: tid=0 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 client.14973 shutdown 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 msgr2=0x55b6bed1e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4007f00 tx=0x7f5dd40077c0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 shutdown_connections 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6bed24ae0 0x55b6bed1e520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b6becff0b0 0x55b6bed1daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55b6bed86790 0x55b6bed854a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 --2- 172.21.15.37:0/2174524936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b6bed0a880 0x55b6bed1dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dc4ff9700 0 client.14973 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.661+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 >> 172.21.15.37:0/2174524936 conn(0x55b6bec2c840 msgr2=0x55b6bed07200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 shutdown_connections 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5dee200b80 1 -- 172.21.15.37:0/2174524936 wait complete. 2024-04-25T21:07:28.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5dee200b80 20 client.14973 trim_cache size 0 max 16384 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24457, 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 130.917058664, 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:07:28.669 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:07:28.670 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 178.668941346 2024-04-25T21:07:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:07:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 msgr2=0x7f5dc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 0x7f5dc80a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00b5c0 tx=0x7f5dcc005fb0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 msgr2=0x7f5ddc101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c9b0 tx=0x7f5dd800cd70 comp rx=0 tx=0).stop 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 shutdown_connections 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5dc80a3c40 0x7f5dc80a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ddc108a30 0x7f5ddc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc06b170 0x7f5ddc101a40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 --2- 172.21.15.37:0/530466238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ddc06bb10 0x7f5ddc101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 >> 172.21.15.37:0/530466238 conn(0x7f5ddc0788f0 msgr2=0x7f5ddc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 shutdown_connections 2024-04-25T21:07:28.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:28.662+0000 7f5de3977700 1 -- 172.21.15.37:0/530466238 wait complete. 2024-04-25T21:07:29.367 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T21:07:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:29 smithi177 ceph-mon[33043]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:29 smithi098 ceph-mon[32569]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:29.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:29 smithi037 ceph-mon[29195]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 -- 172.21.15.37:0/2320282957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c105860 msgr2=0x7fbf5c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 --2- 172.21.15.37:0/2320282957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c105860 0x7fbf5c1098b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf500099c0 tx=0x7fbf50009cd0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 -- 172.21.15.37:0/2320282957 shutdown_connections 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 --2- 172.21.15.37:0/2320282957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c109f90 0x7fbf5c111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 --2- 172.21.15.37:0/2320282957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c104eb0 0x7fbf5c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 --2- 172.21.15.37:0/2320282957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c105860 0x7fbf5c1098b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 -- 172.21.15.37:0/2320282957 >> 172.21.15.37:0/2320282957 conn(0x7fbf5c100740 msgr2=0x7fbf5c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 -- 172.21.15.37:0/2320282957 shutdown_connections 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.489+0000 7fbf62f40700 1 -- 172.21.15.37:0/2320282957 wait complete. 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 Processor -- start 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 -- start start 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c104eb0 0x7fbf5c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c105860 0x7fbf5c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c114e80 con 0x7fbf5c109f90 2024-04-25T21:07:31.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c114d00 con 0x7fbf5c105860 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf62f40700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c115000 con 0x7fbf5c104eb0 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58572/0 (socket says 172.21.15.37:58572) 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 -- 172.21.15.37:0/3342961398 learned_addr learned my addr 172.21.15.37:0/3342961398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c105860 0x7fbf5c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c104eb0 0x7fbf5c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 -- 172.21.15.37:0/3342961398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c104eb0 msgr2=0x7fbf5c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c104eb0 0x7fbf5c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 -- 172.21.15.37:0/3342961398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c105860 msgr2=0x7fbf5c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c105860 0x7fbf5c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 -- 172.21.15.37:0/3342961398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c1a02d0 con 0x7fbf5c109f90 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.490+0000 7fbf614dd700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400b970 tx=0x7fbf5400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.491+0000 7fbf59ffb700 1 -- 172.21.15.37:0/3342961398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54019070 con 0x7fbf5c109f90 2024-04-25T21:07:31.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.491+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c1a05c0 con 0x7fbf5c109f90 2024-04-25T21:07:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.491+0000 7fbf59ffb700 1 -- 172.21.15.37:0/3342961398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5400cde0 con 0x7fbf5c109f90 2024-04-25T21:07:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.491+0000 7fbf59ffb700 1 -- 172.21.15.37:0/3342961398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54007370 con 0x7fbf5c109f90 2024-04-25T21:07:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.491+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a7e40 con 0x7fbf5c109f90 2024-04-25T21:07:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.494+0000 7fbf59ffb700 1 -- 172.21.15.37:0/3342961398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf5401c070 con 0x7fbf5c109f90 2024-04-25T21:07:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.494+0000 7fbf59ffb700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 0x7fbf440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.494+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 0x7fbf440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.494+0000 7fbf59ffb700 1 -- 172.21.15.37:0/3342961398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbf540cf910 con 0x7fbf5c109f90 2024-04-25T21:07:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 0x7fbf440a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ba60 tx=0x7fbf4c005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 Processor -- start 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 -- start start 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee92490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6dede5c40 con 0x55d6dee929d0 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6ded8c8e0 con 0x55d6dee920b0 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6dede1c00 con 0x55d6dee9d880 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58584/0 (socket says 172.21.15.37:58584) 2024-04-25T21:07:31.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.495+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1757780243 learned_addr learned my addr 172.21.15.37:0/1757780243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf614dd700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee92490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1757780243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 msgr2=0x55d6dee9fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1757780243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 msgr2=0x55d6dee92490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee92490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1757780243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ded506f0 con 0x55d6dee929d0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac9e349edadf1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf42ffd700 1 -- 172.21.15.37:0/1757780243 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50003a60 con 0x55d6dee929d0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf42ffd700 1 -- 172.21.15.37:0/1757780243 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50026070 con 0x55d6dee929d0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.496+0000 7fbf42ffd700 1 -- 172.21.15.37:0/1757780243 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000bb40 con 0x55d6dee929d0 2024-04-25T21:07:31.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1757780243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 msgr2=0x55d6dee9d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50003730 comp rx=0 tx=0).stop 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1757780243 shutdown_connections 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee92490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1757780243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9d340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1757780243 >> 172.21.15.37:0/1757780243 conn(0x55d6dedbf840 msgr2=0x55d6dee8e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1757780243 shutdown_connections 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1757780243 wait complete. 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 -- start start 2024-04-25T21:07:31.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.497+0000 7fbf6d7c9b80 1 Processor -- start 2024-04-25T21:07:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6deea0310 con 0x55d6dee929d0 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6ded9f560 con 0x55d6dee920b0 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6deea0490 con 0x55d6dee9d880 2024-04-25T21:07:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf6d7c9b80 10 client.0 _ll_register_callbacks cb 0x55d6ded7e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58590/0 (socket says 172.21.15.37:58590) 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1531433150 learned_addr learned my addr 172.21.15.37:0/1531433150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf614dd700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1531433150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 msgr2=0x55d6dee9a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 msgr2=0x55d6dee9cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 -- 172.21.15.37:0/1531433150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ded33880 con 0x55d6dee929d0 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098e0 tx=0x7fbf500037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c81cfcceadcb99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.498+0000 7fbf3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf500050f0 con 0x55d6dee929d0 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf50026070 con 0x55d6dee929d0 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbf50005290 con 0x55d6dee929d0 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf50005490 con 0x55d6dee929d0 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 10 client.14985 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6deeaaa80 con 0x55d6dee929d0 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50026070 con 0x55d6dee929d0 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf50032400 con 0x55d6dee929d0 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 10 client.14985 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 20 client.14985 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6deeacb90 0x55d6deeae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 4 client.14985 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.499+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6dedf3430 con 0x55d6deeacb90 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.500+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6deeacb90 0x55d6deeae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.500+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6deeacb90 0x55d6deeae000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=117ba3edd2a08062 server_cookie=9529329132cb771b in_seq=0 out_seq=0 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.500+0000 7fbf3bfff700 10 client.14985 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf3bfff700 1 -- 172.21.15.37:0/1531433150 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6dedf3430 con 0x55d6deeacb90 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf3bfff700 10 client.14985 handle_command_reply: tid=0 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 client.14985 shutdown 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 msgr2=0x55d6dee9a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf500098e0 tx=0x7fbf500037a0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 shutdown_connections 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee9a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6deeacb90 0x55d6deeae000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf3bfff700 0 client.14985 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/1531433150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9a250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 >> 172.21.15.37:0/1531433150 conn(0x55d6dedbf840 msgr2=0x55d6dee8e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 shutdown_connections 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/1531433150 wait complete. 2024-04-25T21:07:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.501+0000 7fbf6d7c9b80 20 client.14985 trim_cache size 0 max 16384 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 Processor -- start 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 -- start start 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6deeac340 con 0x55d6dee929d0 2024-04-25T21:07:31.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6deeaca10 con 0x55d6dee920b0 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6deead0e0 con 0x55d6dee9d880 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 -- 172.21.15.37:0/412902523 learned_addr learned my addr 172.21.15.37:0/412902523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf614dd700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 -- 172.21.15.37:0/412902523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 msgr2=0x55d6dee952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 -- 172.21.15.37:0/412902523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 msgr2=0x55d6dee9a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf5bfff700 1 -- 172.21.15.37:0/412902523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6deea9730 con 0x55d6dee929d0 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf614dd700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.522+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:31.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d0f081ef786295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf427fc700 1 -- 172.21.15.37:0/412902523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50041070 con 0x55d6dee929d0 2024-04-25T21:07:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf427fc700 1 -- 172.21.15.37:0/412902523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50026040 con 0x55d6dee929d0 2024-04-25T21:07:31.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf427fc700 1 -- 172.21.15.37:0/412902523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000f910 con 0x55d6dee929d0 2024-04-25T21:07:31.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/412902523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 msgr2=0x55d6dee9fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf50005e80 comp rx=0 tx=0).stop 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/412902523 shutdown_connections 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6dee9a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/412902523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6dee9fc60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/412902523 >> 172.21.15.37:0/412902523 conn(0x55d6dedbf840 msgr2=0x55d6dee8e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.523+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/412902523 shutdown_connections 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/412902523 wait complete. 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 -- start start 2024-04-25T21:07:31.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 Processor -- start 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6deeb4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee97370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6deeade80 con 0x55d6dee929d0 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6deead7b0 con 0x55d6dee920b0 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6deeae550 con 0x55d6dee9d880 2024-04-25T21:07:31.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf6d7c9b80 10 client.0 _ll_register_callbacks cb 0x55d6ded7e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.524+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58606/0 (socket says 172.21.15.37:58606) 2024-04-25T21:07:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 -- 172.21.15.37:0/2418301140 learned_addr learned my addr 172.21.15.37:0/2418301140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf614dd700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee97370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6deeb4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 -- 172.21.15.37:0/2418301140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 msgr2=0x55d6dee97370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee97370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 -- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 msgr2=0x55d6deeb4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6deeb4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 -- 172.21.15.37:0/2418301140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6dee978b0 con 0x55d6dee929d0 2024-04-25T21:07:31.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf614dd700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee97370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf5bfff700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009d10 tx=0x7fbf50005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda8d8e979d1a3da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50041070 con 0x55d6dee929d0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbf50032c90 con 0x55d6dee929d0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf50026040 con 0x55d6dee929d0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf50032610 con 0x55d6dee929d0 2024-04-25T21:07:31.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf6d7c9b80 10 client.14991 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.525+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6def143f0 con 0x55d6dee929d0 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf50026040 con 0x55d6dee929d0 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf5003d480 con 0x55d6dee929d0 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 10 client.14991 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:07:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 20 client.14991 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6def18a00 0x55d6def17710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 4 client.14991 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "status"}) -- 0x55d6deded4d0 con 0x55d6def18a00 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.526+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6def18a00 0x55d6def17710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf60cdc700 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6def18a00 0x55d6def17710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0a5847702a495e8 server_cookie=d71fec8985482966 in_seq=0 out_seq=0 2024-04-25T21:07:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf3bfff700 10 client.14991 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf3bfff700 1 -- 172.21.15.37:0/2418301140 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d6deded4d0 con 0x55d6def18a00 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf3bfff700 10 client.14991 handle_command_reply: tid=0 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 client.14991 shutdown 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 msgr2=0x55d6deeb53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009d10 tx=0x7fbf50005a10 comp rx=0 tx=0).stop 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 shutdown_connections 2024-04-25T21:07:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6dee9d880 0x55d6dee97370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6dee920b0 0x55d6deeb4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x55d6def18a00 0x55d6def17710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 --2- 172.21.15.37:0/2418301140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6dee929d0 0x55d6deeb53b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 >> 172.21.15.37:0/2418301140 conn(0x55d6dedbf840 msgr2=0x55d6deea8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf3bfff700 0 client.14991 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 shutdown_connections 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 1 -- 172.21.15.37:0/2418301140 wait complete. 2024-04-25T21:07:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.527+0000 7fbf6d7c9b80 20 client.14991 trim_cache size 0 max 16384 2024-04-25T21:07:31.538 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:07:31.538 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:07:31.538 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-25T21:07:31.538 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-04-25T21:07:31.538 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 134.787987488, 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 165.29972691699999 2024-04-25T21:07:31.539 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 msgr2=0x7fbf440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 0x7fbf440a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00ba60 tx=0x7fbf4c005d50 comp rx=0 tx=0).stop 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 msgr2=0x7fbf5c19fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400b970 tx=0x7fbf5400bc80 comp rx=0 tx=0).stop 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 shutdown_connections 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf5c109f90 0x7fbf5c19fb90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf5c104eb0 0x7fbf5c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbf440a3c40 0x7fbf440a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 --2- 172.21.15.37:0/3342961398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf5c105860 0x7fbf5c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 >> 172.21.15.37:0/3342961398 conn(0x7fbf5c100740 msgr2=0x7fbf5c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 shutdown_connections 2024-04-25T21:07:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:31.528+0000 7fbf62f40700 1 -- 172.21.15.37:0/3342961398 wait complete. 2024-04-25T21:07:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:31 smithi177 ceph-mon[33043]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:31 smithi098 ceph-mon[32569]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:31.731 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:31 smithi037 ceph-mon[29195]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:32.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T21:07:32.603 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:33.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:33 smithi037 ceph-mon[29195]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:33.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:33 smithi177 ceph-mon[33043]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:33 smithi098 ceph-mon[32569]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.424+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2183763679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78101960 msgr2=0x7fbc78101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.424+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2183763679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78101960 0x7fbc78101de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc64009990 tx=0x7fbc64009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2183763679 shutdown_connections 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2183763679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78102320 0x7fbc7810e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2183763679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78107960 0x7fbc78107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2183763679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78101960 0x7fbc78101de0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2183763679 >> 172.21.15.37:0/2183763679 conn(0x7fbc780789e0 msgr2=0x7fbc78078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2183763679 shutdown_connections 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2183763679 wait complete. 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 Processor -- start 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- start start 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78101960 0x7fbc781990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78102320 0x7fbc78199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111ba0 con 0x7fbc78107960 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111a20 con 0x7fbc78101960 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.425+0000 7fbc7ece1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbc78111d20 con 0x7fbc78102320 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58626/0 (socket says 172.21.15.37:58626) 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2631229418 learned_addr learned my addr 172.21.15.37:0/2631229418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78101960 0x7fbc781990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78102320 0x7fbc78199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2631229418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78102320 msgr2=0x7fbc78199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78102320 0x7fbc78199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2631229418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78101960 msgr2=0x7fbc781990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78101960 0x7fbc781990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2631229418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7819e100 con 0x7fbc78107960 2024-04-25T21:07:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007ee0 tx=0x7fbc7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc75ffb700 1 -- 172.21.15.37:0/2631229418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70004bb0 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc75ffb700 1 -- 172.21.15.37:0/2631229418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc70004d50 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7819e3f0 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc75ffb700 1 -- 172.21.15.37:0/2631229418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70005920 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.426+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781a5c70 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.428+0000 7fbc75ffb700 1 -- 172.21.15.37:0/2631229418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc7001a020 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.429+0000 7fbc75ffb700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 0x7fbc680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.429+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 0x7fbc680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.429+0000 7fbc75ffb700 1 -- 172.21.15.37:0/2631229418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbc700cfa40 con 0x7fbc78107960 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.429+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 0x7fbc680a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005950 tx=0x7fbc6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 Processor -- start 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 -- start start 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba2f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba3a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcfb982c40 con 0x55bcfba3a880 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcfb9298e0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bcfb97ec00 con 0x55bcfba2f9d0 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58628/0 (socket says 172.21.15.37:58628) 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.430+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2884658275 learned_addr learned my addr 172.21.15.37:0/2884658275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba2f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba3a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2884658275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 msgr2=0x55bcfba3a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba3a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2884658275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 msgr2=0x55bcfba2f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba2f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 -- 172.21.15.37:0/2884658275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfb8ed6f0 con 0x55bcfba3a880 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba3a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba2f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc70098190 tx=0x7fbc7009b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd515a26daa9da40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc62ffd700 1 -- 172.21.15.37:0/2884658275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7009bd30 con 0x55bcfba3a880 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc62ffd700 1 -- 172.21.15.37:0/2884658275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc700a00e0 con 0x55bcfba3a880 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.431+0000 7fbc62ffd700 1 -- 172.21.15.37:0/2884658275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7009c220 con 0x55bcfba3a880 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2884658275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 msgr2=0x55bcfba3cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc70098190 tx=0x7fbc7009b710 comp rx=0 tx=0).stop 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2884658275 shutdown_connections 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba3cc70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba2f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2884658275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba3a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2884658275 >> 172.21.15.37:0/2884658275 conn(0x55bcfb95c840 msgr2=0x55bcfba2b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2884658275 shutdown_connections 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2884658275 wait complete. 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 -- start start 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.432+0000 7fbc8956ab80 1 Processor -- start 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f9d0 0x55bcfba37330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba3a880 0x55bcfba37870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba3d370 con 0x55bcfba2f0b0 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcfb93c560 con 0x55bcfba2f9d0 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba3d4f0 con 0x55bcfba3a880 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc8956ab80 10 client.0 _ll_register_callbacks cb 0x55bcfb91b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58642/0 (socket says 172.21.15.37:58642) 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/969807515 learned_addr learned my addr 172.21.15.37:0/969807515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:34.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc77fff700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f9d0 0x55bcfba37330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba3a880 0x55bcfba37870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba3a880 msgr2=0x55bcfba37870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba3a880 0x55bcfba37870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/969807515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f9d0 msgr2=0x55bcfba37330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f9d0 0x55bcfba37330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/969807515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfb8d0880 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c011740 tx=0x7fbc6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f21a25bb5d0715e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc57fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.433+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c004a30 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbc6c004bd0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc700a00e0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c004dd0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 10 client.15003 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcfba47c20 con 0x55bcfba2f0b0 2024-04-25T21:07:34.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc700a00e0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc6c01fa20 con 0x55bcfba2f0b0 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 10 client.15003 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 10 client.15003 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 20 client.15003 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfba4a270 0x55bcfba4b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 4 client.15003 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.434+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcfb990430 con 0x55bcfba4a270 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.435+0000 7fbc77fff700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfba4a270 0x55bcfba4b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.435+0000 7fbc77fff700 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfba4a270 0x55bcfba4b6e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7e9df2c36d1c6bd server_cookie=34be1cc387ad1469 in_seq=0 out_seq=0 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.435+0000 7fbc57fff700 10 client.15003 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc57fff700 1 -- 172.21.15.37:0/969807515 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcfb990430 con 0x55bcfba4a270 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc57fff700 10 client.15003 handle_command_reply: tid=0 2024-04-25T21:07:34.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 client.15003 shutdown 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 msgr2=0x55bcfba3a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c011740 tx=0x7fbc6c0047c0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 shutdown_connections 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfba4a270 0x55bcfba4b6e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba3a880 0x55bcfba37870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3a030 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/969807515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f9d0 0x55bcfba37330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc57fff700 0 client.15003 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 >> 172.21.15.37:0/969807515 conn(0x55bcfb95c840 msgr2=0x55bcfba2b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 shutdown_connections 2024-04-25T21:07:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 1 -- 172.21.15.37:0/969807515 wait complete. 2024-04-25T21:07:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.436+0000 7fbc8956ab80 20 client.15003 trim_cache size 0 max 16384 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.457+0000 7fbc8956ab80 1 Processor -- start 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.457+0000 7fbc8956ab80 1 -- start start 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba3a880 0x55bcfba54b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba49a20 con 0x55bcfba2f9d0 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba4a0f0 con 0x55bcfba3a880 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba4a7c0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58652/0 (socket says 172.21.15.37:58652) 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 -- 172.21.15.37:0/2326012986 learned_addr learned my addr 172.21.15.37:0/2326012986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba3a880 0x55bcfba54b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 -- 172.21.15.37:0/2326012986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f0b0 msgr2=0x55bcfba3cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 -- 172.21.15.37:0/2326012986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba3a880 msgr2=0x55bcfba54b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba3a880 0x55bcfba54b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 -- 172.21.15.37:0/2326012986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfba30b00 con 0x55bcfba2f9d0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc77fff700 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb719d7332836f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.458+0000 7fbc627fc700 1 -- 172.21.15.37:0/2326012986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64003a60 con 0x55bcfba2f9d0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc627fc700 1 -- 172.21.15.37:0/2326012986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64005580 con 0x55bcfba2f9d0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc627fc700 1 -- 172.21.15.37:0/2326012986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400bbd0 con 0x55bcfba2f9d0 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2326012986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 msgr2=0x55bcfba39b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc64005960 comp rx=0 tx=0).stop 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2326012986 shutdown_connections 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba3a880 0x55bcfba54b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f0b0 0x55bcfba3cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/2326012986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba2f9d0 0x55bcfba39b70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2326012986 >> 172.21.15.37:0/2326012986 conn(0x55bcfb95c840 msgr2=0x55bcfba35c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2326012986 shutdown_connections 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.459+0000 7fbc8956ab80 1 -- 172.21.15.37:0/2326012986 wait complete. 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 -- start start 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 Processor -- start 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba34910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba4b560 con 0x55bcfba3a880 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba4ae90 con 0x55bcfba2f0b0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bcfba4bc30 con 0x55bcfba2f9d0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.460+0000 7fbc8956ab80 10 client.0 _ll_register_callbacks cb 0x55bcfb91b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:48080/0 (socket says 172.21.15.37:48080) 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/344259013 learned_addr learned my addr 172.21.15.37:0/344259013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7d27e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58660/0 (socket says 172.21.15.37:58660) 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc77fff700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba34910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 msgr2=0x55bcfba34910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba34910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/344259013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 msgr2=0x55bcfba53080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 -- 172.21.15.37:0/344259013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfba535c0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7d27e700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc7ca7d700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c010760 tx=0x7fbc6c004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1d14546bd1dcb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc57fff700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c00b870 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.461+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc64005580 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fbc6c01f640 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc6c01f840 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 10 client.24722 fetch_fsmap learned FSMap version 26 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcfbab1a60 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc64005580 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc6c0050e0 con 0x55bcfba2f0b0 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 10 client.24722 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:07:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 10 client.24722 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 20 client.24722 populate_metadata read hostname 'smithi037' 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfbab6700 0x55bcfbab5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 4 client.24722 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.462+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "status"}) -- 0x55bcfb98a4d0 con 0x55bcfbab6700 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc77fff700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfbab6700 0x55bcfbab5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc77fff700 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfbab6700 0x55bcfbab5410 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4958682174781b server_cookie=58cc2d272621a59a in_seq=0 out_seq=0 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc57fff700 10 client.24722 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc57fff700 1 -- 172.21.15.37:0/344259013 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bcfb98a4d0 con 0x55bcfbab6700 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc57fff700 10 client.24722 handle_command_reply: tid=0 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 client.24722 shutdown 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 msgr2=0x55bcfba343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c010760 tx=0x7fbc6c004d00 comp rx=0 tx=0).stop 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 shutdown_connections 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55bcfbab6700 0x55bcfbab5410 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcfba3a880 0x55bcfba53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcfba2f0b0 0x55bcfba343d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc57fff700 0 client.24722 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 --2- 172.21.15.37:0/344259013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bcfba2f9d0 0x55bcfba34910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.463+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 >> 172.21.15.37:0/344259013 conn(0x55bcfb95c840 msgr2=0x55bcfba322c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 shutdown_connections 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc8956ab80 1 -- 172.21.15.37:0/344259013 wait complete. 2024-04-25T21:07:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc8956ab80 20 client.24722 trim_cache size 0 max 16384 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24469, 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 149.801585622, 2024-04-25T21:07:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 178.415128748 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 msgr2=0x7fbc680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 0x7fbc680a6180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c005950 tx=0x7fbc6c00b410 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 msgr2=0x7fbc7819d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc70007ee0 tx=0x7fbc7000d3b0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 shutdown_connections 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc78107960 0x7fbc7819d9c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fbc680a3cc0 0x7fbc680a6180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc78101960 0x7fbc781990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 --2- 172.21.15.37:0/2631229418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc78102320 0x7fbc78199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 >> 172.21.15.37:0/2631229418 conn(0x7fbc780789e0 msgr2=0x7fbc7806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.464+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 shutdown_connections 2024-04-25T21:07:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:34.465+0000 7fbc7ece1700 1 -- 172.21.15.37:0/2631229418 wait complete. 2024-04-25T21:07:35.300 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T21:07:35.300 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T21:07:35.310 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:07:35.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T21:07:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:35.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:35 smithi177 ceph-mon[33043]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:35.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:35 smithi098 ceph-mon[32569]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:35.686 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:35 smithi037 ceph-mon[29195]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- 172.21.15.37:0/2586825281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc102aa0 msgr2=0x7f0adc102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2586825281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc102aa0 0x7f0adc102ec0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0099c0 tx=0x7f0acc009cd0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- 172.21.15.37:0/2586825281 shutdown_connections 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2586825281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc104640 0x7f0adc10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2586825281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc102aa0 0x7f0adc102ec0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2586825281 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 0x7f0adc104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- 172.21.15.37:0/2586825281 >> 172.21.15.37:0/2586825281 conn(0x7f0adc0fe040 msgr2=0x7f0adc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- 172.21.15.37:0/2586825281 shutdown_connections 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- 172.21.15.37:0/2586825281 wait complete. 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 Processor -- start 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.348+0000 7f0ae2937700 1 -- start start 2024-04-25T21:07:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc102aa0 0x7f0adc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 0x7f0adc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10e590 con 0x7f0adc104640 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10e410 con 0x7f0adc102aa0 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae2937700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0adc10e710 con 0x7f0adc103ca0 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58688/0 (socket says 172.21.15.37:58688) 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 -- 172.21.15.37:0/2788842498 learned_addr learned my addr 172.21.15.37:0/2788842498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0adbfff700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc102aa0 0x7f0adc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0adb7fe700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 0x7f0adc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 -- 172.21.15.37:0/2788842498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 msgr2=0x7f0adc074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 0x7f0adc074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 -- 172.21.15.37:0/2788842498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc102aa0 msgr2=0x7f0adc0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc102aa0 0x7f0adc0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.349+0000 7f0ae0ed4700 1 -- 172.21.15.37:0/2788842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0adc075c20 con 0x7f0adc104640 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ae0ed4700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ada0 tx=0x7f0ad400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4017020 con 0x7f0adc104640 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad400ce30 con 0x7f0adc104640 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0adc1a2e50 con 0x7f0adc104640 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4005350 con 0x7f0adc104640 2024-04-25T21:07:37.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.350+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0adc1a32b0 con 0x7f0adc104640 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.351+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0adc069730 con 0x7f0adc104640 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.352+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0ad400c980 con 0x7f0adc104640 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.353+0000 7f0ad97fa700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 0x7f0ac40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.353+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0ad40d0020 con 0x7f0adc104640 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.353+0000 7f0adbfff700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 0x7f0ac40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:37.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.353+0000 7f0adbfff700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 0x7f0ac40a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0038c0 tx=0x7f0acc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:37.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.356+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ad4021020 con 0x7f0adc104640 2024-04-25T21:07:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:37 smithi037 ceph-mon[29195]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.583+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0adc062e80 con 0x7f0ac40a3cc0 2024-04-25T21:07:37.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.584+0000 7f0ad97fa700 1 -- 172.21.15.37:0/2788842498 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0adc062e80 con 0x7f0ac40a3cc0 2024-04-25T21:07:37.584 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-04-25T21:07:37.584 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-04-25T21:07:37.585 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-04-25T21:07:37.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 msgr2=0x7f0ac40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:37.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 0x7f0ac40a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0038c0 tx=0x7f0acc00b540 comp rx=0 tx=0).stop 2024-04-25T21:07:37.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 msgr2=0x7f0adc075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ada0 tx=0x7f0ad400c330 comp rx=0 tx=0).stop 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 shutdown_connections 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0adc104640 0x7f0adc075370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0ac40a3cc0 0x7f0ac40a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0adc102aa0 0x7f0adc0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 --2- 172.21.15.37:0/2788842498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc103ca0 0x7f0adc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 >> 172.21.15.37:0/2788842498 conn(0x7f0adc0fe040 msgr2=0x7f0adc1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 shutdown_connections 2024-04-25T21:07:37.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:37.586+0000 7f0ae2937700 1 -- 172.21.15.37:0/2788842498 wait complete. 2024-04-25T21:07:37.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:37 smithi177 ceph-mon[33043]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:37 smithi098 ceph-mon[32569]: pgmap v381: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:38.244 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T21:07:38.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:38 smithi037 ceph-mon[29195]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:38.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:38 smithi098 ceph-mon[32569]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:38.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:38 smithi177 ceph-mon[33043]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:39.649 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:39 smithi037 ceph-mon[29195]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:39.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:39 smithi098 ceph-mon[32569]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:39.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:39 smithi177 ceph-mon[33043]: pgmap v382: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:40.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 -- 172.21.15.37:0/2309192662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 msgr2=0x7f2364109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:40.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 --2- 172.21.15.37:0/2309192662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f2364109df0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b0d0 tx=0x7f235c00b3e0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 -- 172.21.15.37:0/2309192662 shutdown_connections 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 --2- 172.21.15.37:0/2309192662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f2364109df0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 --2- 172.21.15.37:0/2309192662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23641019a0 0x7f2364101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 --2- 172.21.15.37:0/2309192662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2364102ba0 0x7f2364103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 -- 172.21.15.37:0/2309192662 >> 172.21.15.37:0/2309192662 conn(0x7f23640fcf40 msgr2=0x7f23640ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 -- 172.21.15.37:0/2309192662 shutdown_connections 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.205+0000 7f236aeb8700 1 -- 172.21.15.37:0/2309192662 wait complete. 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 Processor -- start 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 -- start start 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 0x7f2364105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 0x7f2364103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:40.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f236410d440 con 0x7f2364103540 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f236410d2c0 con 0x7f2364102ba0 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f236aeb8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f236410d5c0 con 0x7f23641019a0 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37088/0 (socket says 172.21.15.37:37088) 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 -- 172.21.15.37:0/673563832 learned_addr learned my addr 172.21.15.37:0/673563832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2368c54700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 0x7f2364105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2363fff700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 0x7f2364103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 -- 172.21.15.37:0/673563832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 msgr2=0x7f2364105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 0x7f2364105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 -- 172.21.15.37:0/673563832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 msgr2=0x7f2364103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:40.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 0x7f2364103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 -- 172.21.15.37:0/673563832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2364104d70 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2368c54700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 0x7f2364105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2363fff700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 0x7f2364103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2369455700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d010 tx=0x7f235c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.206+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c021070 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.207+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23641a2d10 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.207+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235c01db00 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.207+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c01bca0 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.207+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a31c0 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.208+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2364069730 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.209+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f235c024070 con 0x7f2364103540 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.209+0000 7f2361ffb700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.210+0000 7f2368c54700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.210+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f235c00f020 con 0x7f2364103540 2024-04-25T21:07:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.210+0000 7f2368c54700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 0x7f234c0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2354005950 tx=0x7f23540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.214+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f235c02b020 con 0x7f2364103540 2024-04-25T21:07:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.432+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f23640ff170 con 0x7f234c0a3c40 2024-04-25T21:07:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.446+0000 7f2361ffb700 1 -- 172.21.15.37:0/673563832 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f23640ff170 con 0x7f234c0a3c40 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (2m) 38s ago 2m 18.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d73561b617b3 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi098 running (2m) 40s ago 2m 21.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5be65f57b5a3 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi177 running (3m) 40s ago 3m 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3770701c3923 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (3m) 38s ago 3m 13.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bbdc59c9a89a 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi098 running (3m) 40s ago 3m 18.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 881fea7f9c98 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi177 running (3m) 40s ago 3m 20.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c1e687316958 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (3m) 38s ago 3m 19.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 32bd1f352d9a 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi098 running (3m) 40s ago 3m 18.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1e1d77ab28c6 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi177 running (3m) 40s ago 3m 28.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3e478684a036 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (3m) 38s ago 3m 14.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7511b9ccc433 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi098 running (3m) 40s ago 3m 21.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e3ed99f7701b 2024-04-25T21:07:40.447 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi177 running (3m) 40s ago 3m 20.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 29c912c2056b 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (11m) 38s ago 11m 516M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4401a6809afe 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi098 *:8443,8765 running (8m) 40s ago 8m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6ccdb20392c9 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi177 *:8443,8765 running (8m) 40s ago 8m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 68329c565373 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (11m) 38s ago 11m 53.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 08c86468669f 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi098 running (9m) 40s ago 9m 40.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ff3fce8ce874 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi177 running (9m) 40s ago 9m 42.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 662780321a8a 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (8m) 38s ago 8m 48.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 c95e96414b6b 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi098 running (8m) 40s ago 8m 45.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ee0e29750080 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi177 running (7m) 40s ago 7m 57.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 13e2bbc1ba6b 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (7m) 38s ago 7m 57.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 2532cb3c42d4 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi098 running (6m) 40s ago 6m 44.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8014a000145e 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi177 running (6m) 40s ago 6m 55.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 1fad2a15e935 2024-04-25T21:07:40.448 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (6m) 38s ago 6m 53.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 03253d294e21 2024-04-25T21:07:40.449 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi098 running (5m) 40s ago 5m 42.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ce430d8af64d 2024-04-25T21:07:40.449 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi177 running (5m) 40s ago 5m 53.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 022c6212544b 2024-04-25T21:07:40.449 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (4m) 38s ago 4m 42.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 55cc52ec43b7 2024-04-25T21:07:40.449 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi098 running (4m) 40s ago 4m 48.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3b717ebaf07b 2024-04-25T21:07:40.449 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi177 running (3m) 40s ago 3m 51.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 62a6ac73c676 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 msgr2=0x7f234c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 0x7f234c0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2354005950 tx=0x7f23540058e0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 msgr2=0x7f23641044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d010 tx=0x7f235c007ad0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 shutdown_connections 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2364103540 0x7f23641044c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23641019a0 0x7f2364105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 --2- 172.21.15.37:0/673563832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2364102ba0 0x7f2364103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 >> 172.21.15.37:0/673563832 conn(0x7f23640fcf40 msgr2=0x7f23640fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 shutdown_connections 2024-04-25T21:07:40.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:40.448+0000 7f236aeb8700 1 -- 172.21.15.37:0/673563832 wait complete. 2024-04-25T21:07:41.169 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T21:07:41.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:41.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:41 smithi037 ceph-mon[29195]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:41.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:41 smithi037 ceph-mon[29195]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:41 smithi098 ceph-mon[32569]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:41 smithi098 ceph-mon[32569]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:41 smithi177 ceph-mon[33043]: pgmap v383: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:41.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:41 smithi177 ceph-mon[33043]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1878103525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad81019a0 msgr2=0x7f9ad8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1878103525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad81019a0 0x7f9ad8101dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80099c0 tx=0x7f9ac8009cd0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1878103525 shutdown_connections 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1878103525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad8103540 0x7f9ad8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1878103525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad81019a0 0x7f9ad8101dc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1878103525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8102ba0 0x7f9ad8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1878103525 >> 172.21.15.37:0/1878103525 conn(0x7f9ad80fcf40 msgr2=0x7f9ad80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1878103525 shutdown_connections 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.231+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1878103525 wait complete. 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 Processor -- start 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 -- start start 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad81019a0 0x7f9ad8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8103540 0x7f9ad81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d430 con 0x7f9ad8102ba0 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d2b0 con 0x7f9ad8103540 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adf0ba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad810d5b0 con 0x7f9ad81019a0 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9ad7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9ad7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37110/0 (socket says 172.21.15.37:37110) 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9ad7fff700 1 -- 172.21.15.37:0/1266200835 learned_addr learned my addr 172.21.15.37:0/1266200835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9adce56700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad81019a0 0x7f9ad8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.232+0000 7f9add657700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8103540 0x7f9ad81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 -- 172.21.15.37:0/1266200835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad81019a0 msgr2=0x7f9ad8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad81019a0 0x7f9ad8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 -- 172.21.15.37:0/1266200835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8103540 msgr2=0x7f9ad81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8103540 0x7f9ad81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 -- 172.21.15.37:0/1266200835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad81a0870 con 0x7f9ad8102ba0 2024-04-25T21:07:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad7fff700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9a0 tx=0x7f9acc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00ca10 con 0x7f9ad8102ba0 2024-04-25T21:07:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9acc00cbb0 con 0x7f9ad8102ba0 2024-04-25T21:07:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad81a0b60 con 0x7f9ad8102ba0 2024-04-25T21:07:43.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc018680 con 0x7f9ad8102ba0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.233+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad81a10f0 con 0x7f9ad8102ba0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.234+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad8069730 con 0x7f9ad8102ba0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.235+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9acc010430 con 0x7f9ad8102ba0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.236+0000 7f9ad5ffb700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 0x7f9ac00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.236+0000 7f9adce56700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 0x7f9ac00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.236+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f9acc0d0520 con 0x7f9ad8102ba0 2024-04-25T21:07:43.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.237+0000 7f9adce56700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 0x7f9ac00a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009990 tx=0x7f9ac8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.241+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9acc09cf10 con 0x7f9ad8102ba0 2024-04-25T21:07:43.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.460+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9ad806ae30 con 0x7f9ac00a3cc0 2024-04-25T21:07:43.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.466+0000 7f9ad5ffb700 1 -- 172.21.15.37:0/1266200835 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9ad806ae30 con 0x7f9ac00a3cc0 2024-04-25T21:07:43.467 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T21:07:43.467 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 43s ago 3m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi098=b;smithi098=e;smithi098=h;smithi098=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2024-04-25T21:07:43.467 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 43s ago 9m smithi037=x;smithi098=y;smithi177=z;count:3 2024-04-25T21:07:43.467 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 43s ago 9m smithi037:172.21.15.37=a;smithi098:172.21.15.98=b;smithi177:172.21.15.177=c;count:3 2024-04-25T21:07:43.467 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 43s ago - 2024-04-25T21:07:43.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 msgr2=0x7f9ac00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 0x7f9ac00a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009990 tx=0x7f9ac8005fb0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 msgr2=0x7f9ad81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9a0 tx=0x7f9acc00ecb0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 shutdown_connections 2024-04-25T21:07:43.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f9ac00a3cc0 0x7f9ac00a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad8103540 0x7f9ad81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ad81019a0 0x7f9ad8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 --2- 172.21.15.37:0/1266200835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad8102ba0 0x7f9ad81991b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 >> 172.21.15.37:0/1266200835 conn(0x7f9ad80fcf40 msgr2=0x7f9ad806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 shutdown_connections 2024-04-25T21:07:43.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:43.468+0000 7f9adf0ba700 1 -- 172.21.15.37:0/1266200835 wait complete. 2024-04-25T21:07:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:43 smithi098 ceph-mon[32569]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:43.695 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:43 smithi037 ceph-mon[29195]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:43 smithi177 ceph-mon[33043]: pgmap v384: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:44.288 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:44.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:44 smithi037 ceph-mon[29195]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:44.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:44 smithi098 ceph-mon[32569]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:44.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:44 smithi177 ceph-mon[33043]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:45.670 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:45 smithi098 ceph-mon[32569]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:45 smithi037 ceph-mon[29195]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:45 smithi177 ceph-mon[33043]: pgmap v385: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:46.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 -- 172.21.15.37:0/1299580026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 msgr2=0x7f5e0410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/1299580026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e0410abf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b0a0 tx=0x7f5dfc00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 -- 172.21.15.37:0/1299580026 shutdown_connections 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/1299580026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e0410abf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/1299580026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04103520 0x7f5e04105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/1299580026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04105e50 0x7f5e04108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.356+0000 7f5e0b28b700 1 -- 172.21.15.37:0/1299580026 >> 172.21.15.37:0/1299580026 conn(0x7f5e040fcf30 msgr2=0x7f5e040ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.357+0000 7f5e0b28b700 1 -- 172.21.15.37:0/1299580026 shutdown_connections 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.357+0000 7f5e0b28b700 1 -- 172.21.15.37:0/1299580026 wait complete. 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.357+0000 7f5e0b28b700 1 Processor -- start 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.357+0000 7f5e0b28b700 1 -- start start 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04103520 0x7f5e04198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04105e50 0x7f5e041990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:46.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410e540 con 0x7f5e04108780 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410e3c0 con 0x7f5e04105e50 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e0b28b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0410e6c0 con 0x7f5e04103520 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37138/0 (socket says 172.21.15.37:37138) 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 -- 172.21.15.37:0/4144385369 learned_addr learned my addr 172.21.15.37:0/4144385369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09027700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04103520 0x7f5e04198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e08826700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04105e50 0x7f5e041990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 -- 172.21.15.37:0/4144385369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04103520 msgr2=0x7f5e04198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04103520 0x7f5e04198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 -- 172.21.15.37:0/4144385369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04105e50 msgr2=0x7f5e041990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04105e50 0x7f5e041990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 -- 172.21.15.37:0/4144385369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e04101310 con 0x7f5e04108780 2024-04-25T21:07:46.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.358+0000 7f5e09828700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b070 tx=0x7f5dfc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.359+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc021070 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.359+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dfc007b80 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.359+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc004770 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.359+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e041015a0 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.359+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e04101a90 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.360+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e04069730 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.361+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5dfc003a00 con 0x7f5e04108780 2024-04-25T21:07:46.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.362+0000 7f5dfa7fc700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 0x7f5df00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:46.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.362+0000 7f5e09027700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 0x7f5df00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:46.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.362+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5dfc00f020 con 0x7f5e04108780 2024-04-25T21:07:46.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.362+0000 7f5e09027700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 0x7f5df00a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e00007c60 tx=0x7f5e000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:46.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.367+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dfc02a020 con 0x7f5e04108780 2024-04-25T21:07:46.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.581+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5e040fe500 con 0x7f5df00a3c40 2024-04-25T21:07:46.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.582+0000 7f5dfa7fc700 1 -- 172.21.15.37:0/4144385369 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5e040fe500 con 0x7f5df00a3c40 2024-04-25T21:07:46.582 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-04-25T21:07:46.582 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-04-25T21:07:46.583 INFO:teuthology.orchestra.run.smithi037.stdout:smithi098 172.21.15.98 2024-04-25T21:07:46.583 INFO:teuthology.orchestra.run.smithi037.stdout:smithi177 172.21.15.177 2024-04-25T21:07:46.583 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 msgr2=0x7f5df00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 0x7f5df00a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5e00007c60 tx=0x7f5e000073d0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 msgr2=0x7f5e041a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc00b070 tx=0x7f5dfc00ce50 comp rx=0 tx=0).stop 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 shutdown_connections 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e04108780 0x7f5e041a0140 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e04103520 0x7f5e04198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.584+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f5df00a3c40 0x7f5df00a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.585+0000 7f5e0b28b700 1 --2- 172.21.15.37:0/4144385369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e04105e50 0x7f5e041990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:46.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.585+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 >> 172.21.15.37:0/4144385369 conn(0x7f5e040fcf30 msgr2=0x7f5e04103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:46.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.585+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 shutdown_connections 2024-04-25T21:07:46.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:46.585+0000 7f5e0b28b700 1 -- 172.21.15.37:0/4144385369 wait complete. 2024-04-25T21:07:47.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T21:07:47.642 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:47 smithi177 ceph-mon[33043]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:47.642 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:47 smithi177 ceph-mon[33043]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:47.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:47 smithi098 ceph-mon[32569]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:47.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:47 smithi098 ceph-mon[32569]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:47.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:47 smithi037 ceph-mon[29195]: pgmap v386: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:47.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:47 smithi037 ceph-mon[29195]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:49.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.535+0000 7f47a107f700 1 -- 172.21.15.37:0/3354793294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c10f080 msgr2=0x7f479c111470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.535+0000 7f47a107f700 1 --2- 172.21.15.37:0/3354793294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c10f080 0x7f479c111470 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f479400b0a0 tx=0x7f479400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 -- 172.21.15.37:0/3354793294 shutdown_connections 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 --2- 172.21.15.37:0/3354793294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c10f080 0x7f479c111470 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 --2- 172.21.15.37:0/3354793294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 0x7f479c074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 --2- 172.21.15.37:0/3354793294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c0754a0 0x7f479c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 -- 172.21.15.37:0/3354793294 >> 172.21.15.37:0/3354793294 conn(0x7f479c0fe040 msgr2=0x7f479c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 -- 172.21.15.37:0/3354793294 shutdown_connections 2024-04-25T21:07:49.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.536+0000 7f47a107f700 1 -- 172.21.15.37:0/3354793294 wait complete. 2024-04-25T21:07:49.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:49 smithi037 ceph-mon[29195]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 Processor -- start 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 -- start start 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 0x7f479c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 0x7f479c1a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f479c114bb0 con 0x7f479c0754a0 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f479c114a30 con 0x7f479c10f080 2024-04-25T21:07:49.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f47a107f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f479c114d30 con 0x7f479c076a70 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f479ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f479ad9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39556/0 (socket says 172.21.15.37:39556) 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f479ad9d700 1 -- 172.21.15.37:0/1644466222 learned_addr learned my addr 172.21.15.37:0/1644466222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.537+0000 7f479a59c700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 0x7f479c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479b59e700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 0x7f479c1a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 -- 172.21.15.37:0/1644466222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 msgr2=0x7f479c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 0x7f479c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 -- 172.21.15.37:0/1644466222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 msgr2=0x7f479c1a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 0x7f479c1a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 -- 172.21.15.37:0/1644466222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479c1a6e70 con 0x7f479c0754a0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479b59e700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 0x7f479c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f479ad9d700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b530 tx=0x7f478c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c015020 con 0x7f479c0754a0 2024-04-25T21:07:49.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479c1a7160 con 0x7f479c0754a0 2024-04-25T21:07:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c011ce0 con 0x7f479c0754a0 2024-04-25T21:07:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c010d40 con 0x7f479c0754a0 2024-04-25T21:07:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.538+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479c1a7770 con 0x7f479c0754a0 2024-04-25T21:07:49.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.539+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479c069730 con 0x7f479c0754a0 2024-04-25T21:07:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.541+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f478c018020 con 0x7f479c0754a0 2024-04-25T21:07:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.541+0000 7f4783fff700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 0x7f47840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.541+0000 7f479a59c700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 0x7f47840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.541+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f478c0d0020 con 0x7f479c0754a0 2024-04-25T21:07:49.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.542+0000 7f479a59c700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 0x7f47840a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f479c0767a0 tx=0x7f479000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:49.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.545+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f478c01f020 con 0x7f479c0754a0 2024-04-25T21:07:49.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:49 smithi098 ceph-mon[32569]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:49.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.763+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f479c062e80 con 0x7f47840a3cc0 2024-04-25T21:07:49.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.764+0000 7f4783fff700 1 -- 172.21.15.37:0/1644466222 <== mgr.14152 v2:172.21.15.37:6800/2407888079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f479c062e80 con 0x7f47840a3cc0 2024-04-25T21:07:49.764 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T21:07:49.764 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T21:07:49.764 INFO:teuthology.orchestra.run.smithi037.stdout:smithi098 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010H400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T21:07:49.764 INFO:teuthology.orchestra.run.smithi037.stdout:smithi177 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233007S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 msgr2=0x7f47840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 0x7f47840a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f479c0767a0 tx=0x7f479000a380 comp rx=0 tx=0).stop 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 msgr2=0x7f479c19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f478c00b530 tx=0x7f478c00b840 comp rx=0 tx=0).stop 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 shutdown_connections 2024-04-25T21:07:49.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f47840a3cc0 0x7f47840a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f479c10f080 0x7f479c1a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c0754a0 0x7f479c19f210 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 --2- 172.21.15.37:0/1644466222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f479c076a70 0x7f479c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.766+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 >> 172.21.15.37:0/1644466222 conn(0x7f479c0fe040 msgr2=0x7f479c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.767+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 shutdown_connections 2024-04-25T21:07:49.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:49.767+0000 7f47a107f700 1 -- 172.21.15.37:0/1644466222 wait complete. 2024-04-25T21:07:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:49 smithi177 ceph-mon[33043]: pgmap v387: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:50.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:50 smithi037 ceph-mon[29195]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:50.582 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T21:07:50.591 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:07:50.591 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T21:07:50.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:50 smithi098 ceph-mon[32569]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:50.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:50 smithi177 ceph-mon[33043]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:51.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:51 smithi037 ceph-mon[29195]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:51 smithi098 ceph-mon[32569]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:51.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:51 smithi177 ceph-mon[33043]: pgmap v388: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:52.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 -- 172.21.15.37:0/2768975142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 msgr2=0x7fb170103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:52.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 --2- 172.21.15.37:0/2768975142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb170103000 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb1640099c0 tx=0x7fb164009cd0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 -- 172.21.15.37:0/2768975142 shutdown_connections 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 --2- 172.21.15.37:0/2768975142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb170103540 0x7fb170109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 --2- 172.21.15.37:0/2768975142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1701019a0 0x7fb170101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 --2- 172.21.15.37:0/2768975142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb170103000 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 -- 172.21.15.37:0/2768975142 >> 172.21.15.37:0/2768975142 conn(0x7fb1700fcf40 msgr2=0x7fb1700ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 -- 172.21.15.37:0/2768975142 shutdown_connections 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.558+0000 7fb174a23700 1 -- 172.21.15.37:0/2768975142 wait complete. 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 Processor -- start 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 -- start start 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1701019a0 0x7fb17006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb170103540 0x7fb1701a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb17010d530 con 0x7fb170102ba0 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb17010d3b0 con 0x7fb170103540 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb174a23700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb17010d6b0 con 0x7fb1701019a0 2024-04-25T21:07:52.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb16dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb16dd9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39576/0 (socket says 172.21.15.37:39576) 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb16dd9b700 1 -- 172.21.15.37:0/1878619601 learned_addr learned my addr 172.21.15.37:0/1878619601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb16ed9d700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb170103540 0x7fb1701a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.559+0000 7fb16e59c700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1701019a0 0x7fb17006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 -- 172.21.15.37:0/1878619601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1701019a0 msgr2=0x7fb17006b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1701019a0 0x7fb17006b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 -- 172.21.15.37:0/1878619601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb170103540 msgr2=0x7fb1701a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb170103540 0x7fb1701a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 -- 172.21.15.37:0/1878619601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1701a0990 con 0x7fb170102ba0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb16dd9b700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b580 tx=0x7fb1640036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164027070 con 0x7fb170102ba0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1701a0c20 con 0x7fb170102ba0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb164022470 con 0x7fb170102ba0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400f6e0 con 0x7fb170102ba0 2024-04-25T21:07:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.560+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1701a1150 con 0x7fb170102ba0 2024-04-25T21:07:52.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.561+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb170069730 con 0x7fb170102ba0 2024-04-25T21:07:52.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.562+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb16402a070 con 0x7fb170102ba0 2024-04-25T21:07:52.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.563+0000 7fb15f7fe700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 0x7fb1580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:52.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.563+0000 7fb16e59c700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 0x7fb1580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:52.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.563+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fb1640d1400 con 0x7fb170102ba0 2024-04-25T21:07:52.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.564+0000 7fb16e59c700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 0x7fb1580a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb160006fd0 tx=0x7fb160008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:52.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.567+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb164013070 con 0x7fb170102ba0 2024-04-25T21:07:52.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.822+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb17002dfb0 con 0x7fb170102ba0 2024-04-25T21:07:52.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.823+0000 7fb15f7fe700 1 -- 172.21.15.37:0/1878619601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3085 (secure 0 0 0) 0x7fb16409dfa0 con 0x7fb170102ba0 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:e26 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 26 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T21:05:03.820512+0000 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-25T21:05:22.903199+0000 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-04-25T21:07:52.823 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 5 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2,3,4 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24469,1=24458,2=24457,3=24445,4=24419} 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-04-25T21:07:52.824 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24469 members: 24458,24469,24457,24445,24419 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{0:24469} state up:active seq 9 addr [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{0:24415} state up:standby-replay seq 1 addr [v2:172.21.15.177:6832/3715259407,v1:172.21.15.177:6833/3715259407] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{1:24458} state up:active seq 9 addr [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{1:24401} state up:standby-replay seq 1 addr [v2:172.21.15.98:6834/785792135,v1:172.21.15.98:6835/785792135] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{2:24457} state up:active seq 13 addr [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{2:24380} state up:standby-replay seq 1 addr [v2:172.21.15.98:6832/3425764534,v1:172.21.15.98:6833/3425764534] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{3:24445} state up:active seq 14 addr [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{3:14574} state up:standby-replay seq 1 addr [v2:172.21.15.37:6840/2041063205,v1:172.21.15.37:6841/2041063205] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{4:24419} state up:active seq 13 addr [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{4:14568} state up:standby-replay seq 1 addr [v2:172.21.15.37:6838/2484283883,v1:172.21.15.37:6839/2484283883] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14508} state up:standby seq 1 addr [v2:172.21.15.37:6834/719027051,v1:172.21.15.37:6835/719027051] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.826 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14520} state up:standby seq 1 addr [v2:172.21.15.37:6836/1767596053,v1:172.21.15.37:6837/1767596053] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 msgr2=0x7fb1580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 0x7fb1580a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb160006fd0 tx=0x7fb160008040 comp rx=0 tx=0).stop 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 msgr2=0x7fb17019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b580 tx=0x7fb1640036f0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 shutdown_connections 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fb1580a3c40 0x7fb1580a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb170103540 0x7fb1701a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1701019a0 0x7fb17006b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 --2- 172.21.15.37:0/1878619601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb170102ba0 0x7fb17019da70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.826+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 >> 172.21.15.37:0/1878619601 conn(0x7fb1700fcf40 msgr2=0x7fb170105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.827+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 shutdown_connections 2024-04-25T21:07:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:52.827+0000 7fb174a23700 1 -- 172.21.15.37:0/1878619601 wait complete. 2024-04-25T21:07:52.828 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:07:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:53 smithi037 ceph-mon[29195]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:53.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:53 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1878619601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T21:07:53.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T21:07:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:53 smithi098 ceph-mon[32569]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:53 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1878619601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T21:07:53.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:53 smithi177 ceph-mon[33043]: pgmap v389: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:53.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:53 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1878619601' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T21:07:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:55.604 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:55 smithi037 ceph-mon[29195]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 -- 172.21.15.37:0/4058147229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102bc0 msgr2=0x7f4140103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 --2- 172.21.15.37:0/4058147229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102bc0 0x7f4140103020 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f41300099c0 tx=0x7f4130009cd0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 -- 172.21.15.37:0/4058147229 shutdown_connections 2024-04-25T21:07:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 --2- 172.21.15.37:0/4058147229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140103560 0x7f4140109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 --2- 172.21.15.37:0/4058147229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41401019c0 0x7f4140101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.634+0000 7f4147514700 1 --2- 172.21.15.37:0/4058147229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102bc0 0x7f4140103020 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- 172.21.15.37:0/4058147229 >> 172.21.15.37:0/4058147229 conn(0x7f41400fcf60 msgr2=0x7f41400ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- 172.21.15.37:0/4058147229 shutdown_connections 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- 172.21.15.37:0/4058147229 wait complete. 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 Processor -- start 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- start start 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 0x7f414019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 0x7f41401a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f414010e5d0 con 0x7f41401019c0 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f414010e450 con 0x7f4140102bc0 2024-04-25T21:07:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f4147514700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f414010e750 con 0x7f4140103560 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.635+0000 7f41452b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39588/0 (socket says 172.21.15.37:39588) 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 -- 172.21.15.37:0/368868423 learned_addr learned my addr 172.21.15.37:0/368868423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4145ab1700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 0x7f41401a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4144aaf700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 0x7f414019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 -- 172.21.15.37:0/368868423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 msgr2=0x7f41401a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 0x7f41401a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 -- 172.21.15.37:0/368868423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 msgr2=0x7f414019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 0x7f414019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 -- 172.21.15.37:0/368868423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41401a2a70 con 0x7f41401019c0 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4145ab1700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 0x7f41401a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4144aaf700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 0x7f414019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:55.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41452b0700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c9a0 tx=0x7f413c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c004220 con 0x7f41401019c0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f413c007d10 con 0x7f41401019c0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41401a2d60 con 0x7f41401019c0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c003e10 con 0x7f41401019c0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.636+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401a3200 con 0x7f41401019c0 2024-04-25T21:07:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.637+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41401a34f0 con 0x7f41401019c0 2024-04-25T21:07:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.638+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f413c007860 con 0x7f41401019c0 2024-04-25T21:07:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.639+0000 7f41367fc700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 0x7f412c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.639+0000 7f4144aaf700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 0x7f412c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.639+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f413c0d02e0 con 0x7f41401019c0 2024-04-25T21:07:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.640+0000 7f4144aaf700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 0x7f412c0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4130003820 tx=0x7f4130005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.643+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f413c014030 con 0x7f41401019c0 2024-04-25T21:07:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:55 smithi098 ceph-mon[32569]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:55 smithi177 ceph-mon[33043]: pgmap v390: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:07:55.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.850+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f41401a36c0 con 0x7f41401019c0 2024-04-25T21:07:55.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.851+0000 7f41367fc700 1 -- 172.21.15.37:0/368868423 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5258 (secure 0 0 0) 0x7f413c09cd50 con 0x7f41401019c0 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 90 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T20:56:30.055661+0000 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-25T21:05:35.806342+0000 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-04-25T21:07:55.851 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/1094372261,v1:172.21.15.37:6803/1094372261] [v2:172.21.15.37:6804/1094372261,v1:172.21.15.37:6805/1094372261] exists,up 50a581ae-b05a-4399-8fc9-4fe21b79d2bf 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6800/1749616660,v1:172.21.15.98:6801/1749616660] [v2:172.21.15.98:6802/1749616660,v1:172.21.15.98:6803/1749616660] exists,up d16bca01-f418-416e-9d6a-57b627ccd028 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/5263269,v1:172.21.15.177:6801/5263269] [v2:172.21.15.177:6802/5263269,v1:172.21.15.177:6803/5263269] exists,up 25aa2e48-4132-499e-8958-d7f5b4638000 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/3382087416,v1:172.21.15.37:6811/3382087416] [v2:172.21.15.37:6812/3382087416,v1:172.21.15.37:6813/3382087416] exists,up dd7eb40c-16dd-48be-b06b-2278b4ea4333 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6808/2300793190,v1:172.21.15.98:6809/2300793190] [v2:172.21.15.98:6810/2300793190,v1:172.21.15.98:6811/2300793190] exists,up 363aa3e4-6b35-48ce-a3a1-7e4b21c47651 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 38 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/4266716104,v1:172.21.15.177:6809/4266716104] [v2:172.21.15.177:6810/4266716104,v1:172.21.15.177:6811/4266716104] exists,up baed219b-bf19-48da-a89d-13bb3ff8485f 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 46 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/3655908250,v1:172.21.15.37:6819/3655908250] [v2:172.21.15.37:6820/3655908250,v1:172.21.15.37:6821/3655908250] exists,up 54c67531-e8c5-4746-bfed-c8c47f4e63de 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6816/3844783138,v1:172.21.15.98:6817/3844783138] [v2:172.21.15.98:6818/3844783138,v1:172.21.15.98:6819/3844783138] exists,up c7826cf0-a44f-4917-bec2-bcb8aa242bd9 2024-04-25T21:07:55.852 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 59 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/396846600,v1:172.21.15.177:6817/396846600] [v2:172.21.15.177:6818/396846600,v1:172.21.15.177:6819/396846600] exists,up 2de6cccb-3606-4ff8-9aaa-fda637fade5f 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/884722198,v1:172.21.15.37:6827/884722198] [v2:172.21.15.37:6828/884722198,v1:172.21.15.37:6829/884722198] exists,up 4fa00cfe-c3d2-4741-a451-b8a4c89b6753 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.98:6824/592671509,v1:172.21.15.98:6825/592671509] [v2:172.21.15.98:6826/592671509,v1:172.21.15.98:6827/592671509] exists,up 3397e1ac-4efe-4cd8-8a27-a2f07662d081 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 80 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6824/2245259996,v1:172.21.15.177:6825/2245259996] [v2:172.21.15.177:6826/2245259996,v1:172.21.15.177:6827/2245259996] exists,up 52308cf8-3492-4a38-b722-87b7bf3fb5c0 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.8 [2,11] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1c [8,11] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.13 [4,9] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.20 [5,9] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.21 [4,6] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.25 [5,9] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.2a [3,8] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.31 [4,8] 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/3898721946 expires 2024-04-26T20:57:34.092925+0000 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/3898721946 expires 2024-04-26T20:57:34.092925+0000 2024-04-25T21:07:55.853 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/4006052172 expires 2024-04-26T20:57:34.092925+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1653551922 expires 2024-04-26T20:57:06.373285+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1698545333 expires 2024-04-26T20:57:34.092925+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/767353771 expires 2024-04-26T20:57:06.373285+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3184535068 expires 2024-04-26T20:57:34.092925+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/1093601384 expires 2024-04-26T20:57:06.373285+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1828557826 expires 2024-04-26T20:57:06.373285+0000 2024-04-25T21:07:55.854 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/1093601384 expires 2024-04-26T20:57:06.373285+0000 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 msgr2=0x7f412c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 0x7f412c0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4130003820 tx=0x7f4130005fd0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 msgr2=0x7f414019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c9a0 tx=0x7f413c00cd60 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 shutdown_connections 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4140103560 0x7f41401a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41401019c0 0x7f414019ae60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f412c0a3cc0 0x7f412c0a6180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 --2- 172.21.15.37:0/368868423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4140102bc0 0x7f414019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.853+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 >> 172.21.15.37:0/368868423 conn(0x7f41400fcf60 msgr2=0x7f414010a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.854+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 shutdown_connections 2024-04-25T21:07:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:55.854+0000 7f4147514700 1 -- 172.21.15.37:0/368868423 wait complete. 2024-04-25T21:07:56.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:56 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/368868423' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T21:07:56.726 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:56 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/368868423' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T21:07:56.762 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T21:07:56.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:07:56.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:56 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/368868423' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T21:07:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:57.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:57 smithi037 ceph-mon[29195]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:57 smithi098 ceph-mon[32569]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:57 smithi177 ceph-mon[33043]: pgmap v391: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:07:58.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 -- 172.21.15.37:0/3499775211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 msgr2=0x7f0f8c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 --2- 172.21.15.37:0/3499775211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c107d70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009990 tx=0x7f0f7c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 -- 172.21.15.37:0/3499775211 shutdown_connections 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 --2- 172.21.15.37:0/3499775211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 0x7f0f8c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 --2- 172.21.15.37:0/3499775211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c107d70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 --2- 172.21.15.37:0/3499775211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 0x7f0f8c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 -- 172.21.15.37:0/3499775211 >> 172.21.15.37:0/3499775211 conn(0x7f0f8c0788f0 msgr2=0x7f0f8c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 -- 172.21.15.37:0/3499775211 shutdown_connections 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 -- 172.21.15.37:0/3499775211 wait complete. 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.821+0000 7f0f914bd700 1 Processor -- start 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 -- start start 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 0x7f0f8c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 0x7f0f8c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111b00 con 0x7f0f8c107990 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111980 con 0x7f0f8c101940 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f914bd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8c111c80 con 0x7f0f8c102300 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55628/0 (socket says 172.21.15.37:55628) 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 -- 172.21.15.37:0/1874759278 learned_addr learned my addr 172.21.15.37:0/1874759278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8a7fc700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 0x7f0f8c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8affd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 0x7f0f8c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 -- 172.21.15.37:0/1874759278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 msgr2=0x7f0f8c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 0x7f0f8c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 -- 172.21.15.37:0/1874759278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 msgr2=0x7f0f8c199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 0x7f0f8c199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.822+0000 7f0f8b7fe700 1 -- 172.21.15.37:0/1874759278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8c1a5540 con 0x7f0f8c107990 2024-04-25T21:07:58.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f8b7fe700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007ee0 tx=0x7f0f8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:58.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84004bb0 con 0x7f0f8c107990 2024-04-25T21:07:58.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8c1a5770 con 0x7f0f8c107990 2024-04-25T21:07:58.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84004d50 con 0x7f0f8c107990 2024-04-25T21:07:58.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84005920 con 0x7f0f8c107990 2024-04-25T21:07:58.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.823+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8c1a5c80 con 0x7f0f8c107990 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.824+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8c109eb0 con 0x7f0f8c107990 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.825+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0f8401a020 con 0x7f0f8c107990 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.826+0000 7f0f73fff700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 0x7f0f740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.826+0000 7f0f8affd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 0x7f0f740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.826+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0f840cf9c0 con 0x7f0f8c107990 2024-04-25T21:07:58.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.826+0000 7f0f8affd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 0x7f0f740a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b5c0 tx=0x7f0f7c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:58.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:58.831+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f840d1e20 con 0x7f0f8c107990 2024-04-25T21:07:59.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.089+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f8c109890 con 0x7f0f8c107990 2024-04-25T21:07:59.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.089+0000 7f0f73fff700 1 -- 172.21.15.37:0/1874759278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7f0f84010070 con 0x7f0f8c107990 2024-04-25T21:07:59.089 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:07:59.089 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:07:59.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 msgr2=0x7f0f740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 0x7f0f740a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b5c0 tx=0x7f0f7c005960 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 msgr2=0x7f0f8c1a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0f84007ee0 tx=0x7f0f8400d3b0 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 shutdown_connections 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8c107990 0x7f0f8c1a5000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0f740a3c40 0x7f0f740a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f8c101940 0x7f0f8c199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 --2- 172.21.15.37:0/1874759278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f8c102300 0x7f0f8c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.091+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 >> 172.21.15.37:0/1874759278 conn(0x7f0f8c0788f0 msgr2=0x7f0f8c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.092+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 shutdown_connections 2024-04-25T21:07:59.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:07:59.092+0000 7f0f914bd700 1 -- 172.21.15.37:0/1874759278 wait complete. 2024-04-25T21:07:59.092 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:07:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:59 smithi098 ceph-mon[32569]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:07:59 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1874759278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:59.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:59 smithi037 ceph-mon[29195]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:59.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:07:59 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1874759278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:59 smithi177 ceph-mon[33043]: pgmap v392: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:07:59.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:07:59 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1874759278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:07:59.854 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:00.106 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:01.582 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:01 smithi037 ceph-mon[29195]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:01 smithi098 ceph-mon[32569]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:01 smithi177 ceph-mon[33043]: pgmap v393: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 -- 172.21.15.37:0/338494161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78108ab0 msgr2=0x7efd78108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 --2- 172.21.15.37:0/338494161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78108ab0 0x7efd78108e90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efd74009990 tx=0x7efd74009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 -- 172.21.15.37:0/338494161 shutdown_connections 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 --2- 172.21.15.37:0/338494161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78103480 0x7efd7810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 --2- 172.21.15.37:0/338494161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78108ab0 0x7efd78108e90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 --2- 172.21.15.37:0/338494161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78102ac0 0x7efd78102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 -- 172.21.15.37:0/338494161 >> 172.21.15.37:0/338494161 conn(0x7efd78079bf0 msgr2=0x7efd7807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 -- 172.21.15.37:0/338494161 shutdown_connections 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.840+0000 7efd7f889700 1 -- 172.21.15.37:0/338494161 wait complete. 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.841+0000 7efd7f889700 1 Processor -- start 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.841+0000 7efd7f889700 1 -- start start 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 0x7efd78075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78108ab0 0x7efd780757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd78112ca0 con 0x7efd78102ac0 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd78112b20 con 0x7efd78103480 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7f889700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd78112e20 con 0x7efd78108ab0 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55646/0 (socket says 172.21.15.37:55646) 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 -- 172.21.15.37:0/583866249 learned_addr learned my addr 172.21.15.37:0/583866249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7ce24700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 0x7efd78075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 -- 172.21.15.37:0/583866249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78108ab0 msgr2=0x7efd780757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78108ab0 0x7efd780757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 -- 172.21.15.37:0/583866249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 msgr2=0x7efd78075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 0x7efd78075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7d625700 1 -- 172.21.15.37:0/583866249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd781a7700 con 0x7efd78102ac0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.842+0000 7efd7ce24700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 0x7efd78075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd7d625700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd74000c00 tx=0x7efd740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74027070 con 0x7efd78102ac0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd781a7990 con 0x7efd78102ac0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd74003fa0 con 0x7efd78102ac0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74022db0 con 0x7efd78102ac0 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.843+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd781a7de0 con 0x7efd78102ac0 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.844+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd781041f0 con 0x7efd78102ac0 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.845+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efd7402a070 con 0x7efd78102ac0 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.846+0000 7efd6e7fc700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 0x7efd640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.846+0000 7efd7ce24700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 0x7efd640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.846+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7efd740d10b0 con 0x7efd78102ac0 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.847+0000 7efd7ce24700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 0x7efd640a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd7807a2f0 tx=0x7efd6800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:01.852+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd74013070 con 0x7efd78102ac0 2024-04-25T21:08:02.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.108+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd7802df80 con 0x7efd78102ac0 2024-04-25T21:08:02.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.109+0000 7efd6e7fc700 1 -- 172.21.15.37:0/583866249 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11923 (secure 0 0 0) 0x7efd7409db30 con 0x7efd78102ac0 2024-04-25T21:08:02.109 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:02.109 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:05:22.903199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24458,24469,24457,24445,24419]},"id":1}]} 2024-04-25T21:08:02.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 msgr2=0x7efd640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:02.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 0x7efd640a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd7807a2f0 tx=0x7efd6800b5f0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 msgr2=0x7efd78074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efd74000c00 tx=0x7efd740047b0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 shutdown_connections 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd78108ab0 0x7efd780757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78102ac0 0x7efd78074d40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efd640a3c40 0x7efd640a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 --2- 172.21.15.37:0/583866249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd78103480 0x7efd78075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 >> 172.21.15.37:0/583866249 conn(0x7efd78079bf0 msgr2=0x7efd78100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 shutdown_connections 2024-04-25T21:08:02.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:02.111+0000 7efd7f889700 1 -- 172.21.15.37:0/583866249 wait complete. 2024-04-25T21:08:02.113 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 26 2024-04-25T21:08:02.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:02 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/583866249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:02.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:02 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/583866249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:02.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:08:02.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:02 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/583866249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:03.016 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:03 smithi098 ceph-mon[32569]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:03.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:03 smithi037 ceph-mon[29195]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:03 smithi177 ceph-mon[33043]: pgmap v394: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:04.579 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:04.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.580 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:04 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:04.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:04 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:04 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 -- 172.21.15.37:0/527886646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81034b0 msgr2=0x7f17d810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 --2- 172.21.15.37:0/527886646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81034b0 0x7f17d810f920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f17d000b0d0 tx=0x7f17d000b3e0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 -- 172.21.15.37:0/527886646 shutdown_connections 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 --2- 172.21.15.37:0/527886646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d81034b0 0x7f17d810f920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 --2- 172.21.15.37:0/527886646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8107940 0x7f17d8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 --2- 172.21.15.37:0/527886646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d8102af0 0x7f17d8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 -- 172.21.15.37:0/527886646 >> 172.21.15.37:0/527886646 conn(0x7f17d8079ba0 msgr2=0x7f17d8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 -- 172.21.15.37:0/527886646 shutdown_connections 2024-04-25T21:08:04.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.833+0000 7f17df25e700 1 -- 172.21.15.37:0/527886646 wait complete. 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 Processor -- start 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 -- start start 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8102af0 0x7f17d8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d81034b0 0x7f17d8075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8112cb0 con 0x7f17d8107940 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8112b30 con 0x7f17d8102af0 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17df25e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17d8112e30 con 0x7f17d81034b0 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17dd7fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17dd7fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55660/0 (socket says 172.21.15.37:55660) 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17dd7fb700 1 -- 172.21.15.37:0/3173293910 learned_addr learned my addr 172.21.15.37:0/3173293910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.834+0000 7f17d7fff700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d81034b0 0x7f17d8075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:04.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dcffa700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8102af0 0x7f17d8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 -- 172.21.15.37:0/3173293910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d81034b0 msgr2=0x7f17d8075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d81034b0 0x7f17d8075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 -- 172.21.15.37:0/3173293910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8102af0 msgr2=0x7f17d8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8102af0 0x7f17d8074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 -- 172.21.15.37:0/3173293910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d81a7670 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17dd7fb700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f17d0007a20 tx=0x7f17d0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d001f070 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d81a7900 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d0003c00 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d001b4a0 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.835+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81a7db0 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.836+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d8069730 con 0x7f17d8107940 2024-04-25T21:08:04.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.837+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f17d0022070 con 0x7f17d8107940 2024-04-25T21:08:04.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.838+0000 7f17d5ffb700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 0x7f17c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:04.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.838+0000 7f17dcffa700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 0x7f17c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:04.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.838+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f17d000f020 con 0x7f17d8107940 2024-04-25T21:08:04.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.839+0000 7f17dcffa700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 0x7f17c00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:04.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:04.847+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17d0029020 con 0x7f17d8107940 2024-04-25T21:08:05.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.056+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17d802df80 con 0x7f17d8107940 2024-04-25T21:08:05.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.058+0000 7f17d5ffb700 1 -- 172.21.15.37:0/3173293910 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22571 (secure 0 0 0) 0x7f17d009dfb0 con 0x7f17d8107940 2024-04-25T21:08:05.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:05.059 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":90,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:05:35.806342+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 msgr2=0x7f17c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 0x7f17c00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f17c8006fd0 tx=0x7f17c8008040 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 msgr2=0x7f17d80757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f17d0007a20 tx=0x7f17d0007a50 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 shutdown_connections 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d8107940 0x7f17d80757c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f17c00a3c40 0x7f17c00a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17d8102af0 0x7f17d8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 --2- 172.21.15.37:0/3173293910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17d81034b0 0x7f17d8075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:05.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.061+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 >> 172.21.15.37:0/3173293910 conn(0x7f17d8079ba0 msgr2=0x7f17d8100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:05.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.062+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 shutdown_connections 2024-04-25T21:08:05.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:05.062+0000 7f17df25e700 1 -- 172.21.15.37:0/3173293910 wait complete. 2024-04-25T21:08:05.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T21:05:03.820512+0000', 'modified': '2024-04-25T21:05:22.903199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24445, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6839/2484283883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 2484283883}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 2484283883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14574': {'gid': 14574, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6841/2041063205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 2041063205}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 2041063205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24380': {'gid': 24380, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6833/3425764534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 3425764534}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 3425764534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/785792135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 785792135}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 785792135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6833/3715259407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3715259407}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 3715259407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.98:6837/2446670971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2446670971}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2446670971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.177:6835/724905952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 724905952}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 724905952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.177:6837/2828024557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 2828024557}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 2828024557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.98:6839/899178427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 899178427}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 899178427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6839/3063347892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3063347892}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3063347892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24458, 24469, 24457, 24445, 24419]} 2024-04-25T21:08:05.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T21:08:05.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3173293910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:05 smithi098 ceph-mon[32569]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3173293910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:05.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:05 smithi037 ceph-mon[29195]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:06.055 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: pgmap v395: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3173293910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:06.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:05 smithi177 ceph-mon[33043]: from='mgr.14152 172.21.15.37:0/1725309741' entity='mgr.x' 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 -- 172.21.15.37:0/3105290783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc04108a70 msgr2=0x7fdc04108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 --2- 172.21.15.37:0/3105290783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc04108a70 0x7fdc04108e50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009990 tx=0x7fdbf4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 -- 172.21.15.37:0/3105290783 shutdown_connections 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 --2- 172.21.15.37:0/3105290783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406bb30 0x7fdc0410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 --2- 172.21.15.37:0/3105290783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc04108a70 0x7fdc04108e50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 --2- 172.21.15.37:0/3105290783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc0406b170 0x7fdc0406b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 -- 172.21.15.37:0/3105290783 >> 172.21.15.37:0/3105290783 conn(0x7fdc040777b0 msgr2=0x7fdc04077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 -- 172.21.15.37:0/3105290783 shutdown_connections 2024-04-25T21:08:07.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.767+0000 7fdc09942700 1 -- 172.21.15.37:0/3105290783 wait complete. 2024-04-25T21:08:07.769 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:07 smithi037 ceph-mon[29195]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.768+0000 7fdc09942700 1 Processor -- start 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 -- start start 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406b170 0x7fdc041990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc04108a70 0x7fdc0419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04112c60 con 0x7fdc0406bb30 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04112ae0 con 0x7fdc0406b170 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc09942700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04112de0 con 0x7fdc04108a70 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55684/0 (socket says 172.21.15.37:55684) 2024-04-25T21:08:07.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 -- 172.21.15.37:0/2858657321 learned_addr learned my addr 172.21.15.37:0/2858657321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406b170 0x7fdc041990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc037fe700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc04108a70 0x7fdc0419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 -- 172.21.15.37:0/2858657321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc04108a70 msgr2=0x7fdc0419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc04108a70 0x7fdc0419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 -- 172.21.15.37:0/2858657321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406b170 msgr2=0x7fdc041990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406b170 0x7fdc041990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.769+0000 7fdc027fc700 1 -- 172.21.15.37:0/2858657321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0419e0e0 con 0x7fdc0406bb30 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800d980 tx=0x7fdbf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc08940700 1 -- 172.21.15.37:0/2858657321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8017070 con 0x7fdc0406bb30 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0419e3d0 con 0x7fdc0406bb30 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc08940700 1 -- 172.21.15.37:0/2858657321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf8005020 con 0x7fdc0406bb30 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc08940700 1 -- 172.21.15.37:0/2858657321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8009b50 con 0x7fdc0406bb30 2024-04-25T21:08:07.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.770+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc041a5c50 con 0x7fdc0406bb30 2024-04-25T21:08:07.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.772+0000 7fdc08940700 1 -- 172.21.15.37:0/2858657321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdbf80051d0 con 0x7fdc0406bb30 2024-04-25T21:08:07.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.773+0000 7fdc08940700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 0x7fdbec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.773+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 0x7fdbec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.773+0000 7fdc08940700 1 -- 172.21.15.37:0/2858657321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fdbf80d0330 con 0x7fdc0406bb30 2024-04-25T21:08:07.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.773+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 0x7fdbec0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400b5c0 tx=0x7fdbf4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 Processor -- start 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 -- start start 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f27d0 0x55bad43f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f7580 0x55bad43f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bad4344ce0 con 0x55bad43f30f0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bad42eb910 con 0x55bad43f7580 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bad4340ca0 con 0x55bad43f27d0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55688/0 (socket says 172.21.15.37:55688) 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 -- 172.21.15.37:0/1510823118 learned_addr learned my addr 172.21.15.37:0/1510823118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f27d0 0x55bad43f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc037fe700 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f7580 0x55bad43f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 -- 172.21.15.37:0/1510823118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f27d0 msgr2=0x55bad43f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f27d0 0x55bad43f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 -- 172.21.15.37:0/1510823118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f7580 msgr2=0x55bad43f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f7580 0x55bad43f79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.774+0000 7fdc027fc700 1 -- 172.21.15.37:0/1510823118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad4292880 con 0x55bad43f30f0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc027fc700 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809c5b0 tx=0x7fdbf809c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8451e6fba8e1ca78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdbe9ffb700 1 -- 172.21.15.37:0/1510823118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80a5290 con 0x55bad43f30f0 2024-04-25T21:08:07.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdbe9ffb700 1 -- 172.21.15.37:0/1510823118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf809a920 con 0x55bad43f30f0 2024-04-25T21:08:07.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdbe9ffb700 1 -- 172.21.15.37:0/1510823118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf80a5020 con 0x55bad43f30f0 2024-04-25T21:08:07.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 -- 172.21.15.37:0/1510823118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 msgr2=0x55bad43f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809c5b0 tx=0x7fdbf809c620 comp rx=0 tx=0).stop 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 -- 172.21.15.37:0/1510823118 shutdown_connections 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f7580 0x55bad43f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f27d0 0x55bad43f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/1510823118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43f7040 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.775+0000 7fdc141cbb80 1 -- 172.21.15.37:0/1510823118 >> 172.21.15.37:0/1510823118 conn(0x55bad431f640 msgr2=0x55bad43ed820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- 172.21.15.37:0/1510823118 shutdown_connections 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- 172.21.15.37:0/1510823118 wait complete. 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- start start 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 Processor -- start 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad43fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bad43ff5b0 con 0x55bad43f30f0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bad42e3b80 con 0x55bad43f27d0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bad43ff730 con 0x55bad43f7580 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.776+0000 7fdc141cbb80 10 client.0 _ll_register_callbacks cb 0x55bad42dd900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55692/0 (socket says 172.21.15.37:55692) 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 -- 172.21.15.37:0/644334183 learned_addr learned my addr 172.21.15.37:0/644334183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc037fe700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad43fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 -- 172.21.15.37:0/644334183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 msgr2=0x55bad43fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad43fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 -- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 msgr2=0x55bad43feab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43feab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 -- 172.21.15.37:0/644334183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad42fc220 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc027fc700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809c320 tx=0x7fdbf80870c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5050f40f244b9251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdbe2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8066060 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fdbf809ca80 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf809a920 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbf80a4e00 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc141cbb80 10 client.15081 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.777+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bad4409e80 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf809a920 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdbf8098370 con 0x55bad43f30f0 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 20 client.15081 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:07.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad440bff0 0x55bad440d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 4 client.15081 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad43524d0 con 0x55bad440bff0 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.778+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad440bff0 0x55bad440d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad440bff0 0x55bad440d460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=567e27f98eed99f0 server_cookie=ce8d2cdef2c879d2 in_seq=0 out_seq=0 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdbe2ffd700 10 client.15081 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/644334183 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bad43524d0 con 0x55bad440bff0 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdbe2ffd700 10 client.15081 handle_command_reply: tid=0 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdc141cbb80 1 client.15081 shutdown 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 msgr2=0x55bad43fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.779+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdbf809c320 tx=0x7fdbf80870c0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 shutdown_connections 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad43fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43feab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad440bff0 0x55bad440d460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdbe2ffd700 0 client.15081 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/644334183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fbdb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 >> 172.21.15.37:0/644334183 conn(0x55bad431f640 msgr2=0x55bad43eca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 shutdown_connections 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 1 -- 172.21.15.37:0/644334183 wait complete. 2024-04-25T21:08:07.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.780+0000 7fdc141cbb80 20 client.15081 trim_cache size 0 max 16384 2024-04-25T21:08:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.800+0000 7fdc141cbb80 1 Processor -- start 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.800+0000 7fdc141cbb80 1 -- start start 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.800+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad44168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bad440b7a0 con 0x55bad43f30f0 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bad440be70 con 0x55bad43f27d0 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bad440c540 con 0x55bad43f7580 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55708/0 (socket says 172.21.15.37:55708) 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 -- 172.21.15.37:0/3035160123 learned_addr learned my addr 172.21.15.37:0/3035160123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 -- 172.21.15.37:0/3035160123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 msgr2=0x55bad44168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc037fe700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad44168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad44168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 -- 172.21.15.37:0/3035160123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 msgr2=0x55bad43f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 -- 172.21.15.37:0/3035160123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad43f2e00 con 0x55bad43f30f0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc037fe700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad44168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdc027fc700 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80a0b90 tx=0x7fdbf809c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551129bbdf52a233 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdbe97fa700 1 -- 172.21.15.37:0/3035160123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8066070 con 0x55bad43f30f0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdbe97fa700 1 -- 172.21.15.37:0/3035160123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf809a920 con 0x55bad43f30f0 2024-04-25T21:08:07.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.801+0000 7fdbe97fa700 1 -- 172.21.15.37:0/3035160123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800ebc0 con 0x55bad43f30f0 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- 172.21.15.37:0/3035160123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 msgr2=0x55bad43fe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80a0b90 tx=0x7fdbf809c620 comp rx=0 tx=0).stop 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- 172.21.15.37:0/3035160123 shutdown_connections 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad44168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f27d0 0x55bad43f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/3035160123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f30f0 0x55bad43fe5f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- 172.21.15.37:0/3035160123 >> 172.21.15.37:0/3035160123 conn(0x55bad431f640 msgr2=0x55bad43f15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- 172.21.15.37:0/3035160123 shutdown_connections 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- 172.21.15.37:0/3035160123 wait complete. 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 -- start start 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.802+0000 7fdc141cbb80 1 Processor -- start 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f30f0 0x55bad4410ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad4411020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bad440d2e0 con 0x55bad43f27d0 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bad440cc10 con 0x55bad43f30f0 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55bad440d9b0 con 0x55bad43f7580 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc141cbb80 10 client.0 _ll_register_callbacks cb 0x55bad42dd900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55724/0 (socket says 172.21.15.37:55724) 2024-04-25T21:08:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 -- 172.21.15.37:0/2403677905 learned_addr learned my addr 172.21.15.37:0/2403677905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f30f0 0x55bad4410ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc037fe700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad4411020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 -- 172.21.15.37:0/2403677905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 msgr2=0x55bad4411020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad4411020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 -- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f30f0 msgr2=0x55bad4410ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f30f0 0x55bad4410ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 -- 172.21.15.37:0/2403677905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad4414fc0 con 0x55bad43f27d0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdc02ffd700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc448a54fe9c49e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdbe2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.803+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4033430 con 0x55bad43f27d0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fdbf40335d0 con 0x55bad43f27d0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf809a920 con 0x55bad43f27d0 2024-04-25T21:08:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdbf40337d0 con 0x55bad43f27d0 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 10 client.15087 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bad4473630 con 0x55bad43f27d0 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf809a920 con 0x55bad43f27d0 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdbf400f460 con 0x55bad43f27d0 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 10 client.15087 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 20 client.15087 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad4478240 0x55bad4476f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 4 client.15087 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "status"}) -- 0x55bad434c570 con 0x55bad4478240 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.804+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad4478240 0x55bad4476f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdbe2ffd700 10 client.15087 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc027fc700 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad4478240 0x55bad4476f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a808b7c0888f58b2 server_cookie=62e283aefede4b43 in_seq=0 out_seq=0 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdbe2ffd700 1 -- 172.21.15.37:0/2403677905 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bad434c570 con 0x55bad4478240 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdbe2ffd700 10 client.15087 handle_command_reply: tid=0 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 client.15087 shutdown 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 msgr2=0x55bad44105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4000c00 tx=0x7fdbf4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 shutdown_connections 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55bad43f7580 0x55bad4411020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55bad4478240 0x55bad4476f50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bad43f27d0 0x55bad44105a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 --2- 172.21.15.37:0/2403677905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bad43f30f0 0x55bad4410ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdbe2ffd700 0 client.15087 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 >> 172.21.15.37:0/2403677905 conn(0x55bad431f640 msgr2=0x55bad43f1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 shutdown_connections 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.805+0000 7fdc141cbb80 1 -- 172.21.15.37:0/2403677905 wait complete. 2024-04-25T21:08:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc141cbb80 20 client.15087 trim_cache size 0 max 16384 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 4, 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24419, 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 168.04030272899999, 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:08:07.812 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:08:07.813 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:08:07.813 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 214.75115029299999 2024-04-25T21:08:07.813 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:07.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 msgr2=0x7fdbec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 0x7fdbec0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400b5c0 tx=0x7fdbf4005960 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 msgr2=0x7fdc04199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800d980 tx=0x7fdbf800dc90 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 shutdown_connections 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdc04108a70 0x7fdc0419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fdbec0a3c40 0x7fdbec0a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc0406b170 0x7fdc041990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 --2- 172.21.15.37:0/2858657321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0406bb30 0x7fdc04199610 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 >> 172.21.15.37:0/2858657321 conn(0x7fdc040777b0 msgr2=0x7fdc04100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 shutdown_connections 2024-04-25T21:08:07.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:07.806+0000 7fdc09942700 1 -- 172.21.15.37:0/2858657321 wait complete. 2024-04-25T21:08:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:07 smithi098 ceph-mon[32569]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:07 smithi177 ceph-mon[33043]: pgmap v396: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:08.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.f status 2024-04-25T21:08:08.788 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:09.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:09 smithi037 ceph-mon[29195]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:09 smithi177 ceph-mon[33043]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:10.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:09 smithi098 ceph-mon[32569]: pgmap v397: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 -- 172.21.15.37:0/3340482910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170102ac0 msgr2=0x7f5170102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 --2- 172.21.15.37:0/3340482910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170102ac0 0x7f5170102f40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f516c009990 tx=0x7f516c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 -- 172.21.15.37:0/3340482910 shutdown_connections 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 --2- 172.21.15.37:0/3340482910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170103480 0x7f517010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 --2- 172.21.15.37:0/3340482910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170108a70 0x7f5170108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 --2- 172.21.15.37:0/3340482910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170102ac0 0x7f5170102f40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 -- 172.21.15.37:0/3340482910 >> 172.21.15.37:0/3340482910 conn(0x7f5170079bb0 msgr2=0x7f5170079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 -- 172.21.15.37:0/3340482910 shutdown_connections 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.586+0000 7f51777f2700 1 -- 172.21.15.37:0/3340482910 wait complete. 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 Processor -- start 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 -- start start 2024-04-25T21:08:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 0x7f5170199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 0x7f517019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5170112c00 con 0x7f5170108a70 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5170112a80 con 0x7f5170103480 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f51777f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5170112d80 con 0x7f5170102ac0 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f5174d8d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f5174d8d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f5174d8d700 1 -- 172.21.15.37:0/4265635973 learned_addr learned my addr 172.21.15.37:0/4265635973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.587+0000 7f5175d8f700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 0x7f517019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f517558e700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 0x7f5170199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 -- 172.21.15.37:0/4265635973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 msgr2=0x7f5170199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 0x7f5170199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 -- 172.21.15.37:0/4265635973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 msgr2=0x7f5170199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 -- 172.21.15.37:0/4265635973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517019e140 con 0x7f5170108a70 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f517558e700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 0x7f5170199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5174d8d700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f5175d8f700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 0x7f517019da00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f516800cc50 tx=0x7f516800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f51627fc700 1 -- 172.21.15.37:0/4265635973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168016020 con 0x7f5170108a70 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517019e430 con 0x7f5170108a70 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f51627fc700 1 -- 172.21.15.37:0/4265635973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5168012420 con 0x7f5170108a70 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f51627fc700 1 -- 172.21.15.37:0/4265635973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680115a0 con 0x7f5170108a70 2024-04-25T21:08:10.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.588+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51701a5cb0 con 0x7f5170108a70 2024-04-25T21:08:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.590+0000 7f51627fc700 1 -- 172.21.15.37:0/4265635973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5168019020 con 0x7f5170108a70 2024-04-25T21:08:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.591+0000 7f51627fc700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.591+0000 7f517558e700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.591+0000 7f51627fc700 1 -- 172.21.15.37:0/4265635973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f51680d0300 con 0x7f5170108a70 2024-04-25T21:08:10.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.591+0000 7f517558e700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 0x7f515c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5164005fd0 tx=0x7f5164005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 Processor -- start 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 -- start start 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0d0990 0x55c38b0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c38b019ce0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c38afc0910 con 0x55c38b0cc500 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f518207bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c38b015ca0 con 0x55c38b0d0990 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f517558e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f517558e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41188/0 (socket says 172.21.15.37:41188) 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f517558e700 1 -- 172.21.15.37:0/4173739606 learned_addr learned my addr 172.21.15.37:0/4173739606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f5174d8d700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.592+0000 7f5175d8f700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0d0990 0x55c38b0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 -- 172.21.15.37:0/4173739606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0d0990 msgr2=0x55c38b0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0d0990 0x55c38b0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 -- 172.21.15.37:0/4173739606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 msgr2=0x55c38b0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 -- 172.21.15.37:0/4173739606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c38af67880 con 0x55c38b0cbbe0 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f5174d8d700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f517558e700 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f516400dbd0 tx=0x7f5164010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d4ab69d9332ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f515b7fe700 1 -- 172.21.15.37:0/4173739606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164020070 con 0x55c38b0cbbe0 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f515b7fe700 1 -- 172.21.15.37:0/4173739606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516400fde0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.593+0000 7f515b7fe700 1 -- 172.21.15.37:0/4173739606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516401b3b0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- 172.21.15.37:0/4173739606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 msgr2=0x55c38b0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f516400dbd0 tx=0x7f5164010000 comp rx=0 tx=0).stop 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- 172.21.15.37:0/4173739606 shutdown_connections 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0d0990 0x55c38b0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0cbbe0 0x55c38b0cbfc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 --2- 172.21.15.37:0/4173739606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- 172.21.15.37:0/4173739606 >> 172.21.15.37:0/4173739606 conn(0x55c38aff4640 msgr2=0x55c38b0c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- 172.21.15.37:0/4173739606 shutdown_connections 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- 172.21.15.37:0/4173739606 wait complete. 2024-04-25T21:08:10.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 -- start start 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.594+0000 7f518207bb80 1 Processor -- start 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0d8a20 con 0x55c38b0d0990 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c38afb8b80 con 0x55c38b0cc500 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0d8ba0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f518207bb80 10 client.0 _ll_register_callbacks cb 0x55c38afb2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41194/0 (socket says 172.21.15.37:41194) 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 -- 172.21.15.37:0/1797454471 learned_addr learned my addr 172.21.15.37:0/1797454471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f517558e700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5174d8d700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 -- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cbbe0 msgr2=0x55c38b0d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 -- 172.21.15.37:0/1797454471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 msgr2=0x55c38b0d51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 -- 172.21.15.37:0/1797454471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c38afd1220 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5175d8f700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f516800cc20 tx=0x7f516809bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1d8ee5e0abb109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680ba060 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.595+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f5168018d80 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516400fde0 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51680a09a0 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 10 client.15099 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c38b0e3330 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516400fde0 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51680a4dd0 con 0x55c38b0d0990 2024-04-25T21:08:10.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 20 client.15099 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b0e58d0 0x55c38b0e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 4 client.15099 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c38b0274d0 con 0x55c38b0e58d0 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.596+0000 7f517558e700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b0e58d0 0x55c38b0e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.597+0000 7f5158ff9700 10 client.15099 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.597+0000 7f517558e700 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b0e58d0 0x55c38b0e6d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f48aa637a33f611 server_cookie=bb96677bb0712b74 in_seq=0 out_seq=0 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f5158ff9700 1 -- 172.21.15.37:0/1797454471 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c38b0274d0 con 0x55c38b0e58d0 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f5158ff9700 10 client.15099 handle_command_reply: tid=0 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 client.15099 shutdown 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 msgr2=0x55c38b0d56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f516800cc20 tx=0x7f516809bfd0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 shutdown_connections 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b0e58d0 0x55c38b0e6d40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d56e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 --2- 172.21.15.37:0/1797454471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cc500 0x55c38b0d51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 >> 172.21.15.37:0/1797454471 conn(0x55c38aff4640 msgr2=0x55c38b0d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f5158ff9700 0 client.15099 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 shutdown_connections 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 1 -- 172.21.15.37:0/1797454471 wait complete. 2024-04-25T21:08:10.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.598+0000 7f518207bb80 20 client.15099 trim_cache size 0 max 16384 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 Processor -- start 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 -- start start 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e5080 con 0x55c38b0d0990 2024-04-25T21:08:10.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e5750 con 0x55c38b0cbbe0 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f518207bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e5e20 con 0x55c38b0cc500 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f5174d8d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41210/0 (socket says 172.21.15.37:41210) 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f5175d8f700 1 -- 172.21.15.37:0/2485124088 learned_addr learned my addr 172.21.15.37:0/2485124088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.619+0000 7f517558e700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 -- 172.21.15.37:0/2485124088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 msgr2=0x55c38b0d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 -- 172.21.15.37:0/2485124088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 msgr2=0x55c38b0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 -- 172.21.15.37:0/2485124088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c38b0d8370 con 0x55c38b0d0990 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5174d8d700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:10.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f517558e700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:10.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f5175d8f700 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51680ce180 tx=0x7f516809cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49db2e82022949ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f515affd700 1 -- 172.21.15.37:0/2485124088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51680ba020 con 0x55c38b0d0990 2024-04-25T21:08:10.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f515affd700 1 -- 172.21.15.37:0/2485124088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5168080020 con 0x55c38b0d0990 2024-04-25T21:08:10.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.620+0000 7f515affd700 1 -- 172.21.15.37:0/2485124088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516809a8f0 con 0x55c38b0d0990 2024-04-25T21:08:10.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- 172.21.15.37:0/2485124088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 msgr2=0x55c38b0c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51680ce180 tx=0x7f516809cc10 comp rx=0 tx=0).stop 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- 172.21.15.37:0/2485124088 shutdown_connections 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0c65e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 --2- 172.21.15.37:0/2485124088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- 172.21.15.37:0/2485124088 >> 172.21.15.37:0/2485124088 conn(0x55c38aff4640 msgr2=0x55c38b0d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- 172.21.15.37:0/2485124088 shutdown_connections 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- 172.21.15.37:0/2485124088 wait complete. 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 -- start start 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.621+0000 7f518207bb80 1 Processor -- start 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e6bc0 con 0x55c38b0d0990 2024-04-25T21:08:10.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e64f0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c38b0e7290 con 0x55c38b0cc500 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f518207bb80 10 client.0 _ll_register_callbacks cb 0x55c38afb2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f5175d8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41214/0 (socket says 172.21.15.37:41214) 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f5175d8f700 1 -- 172.21.15.37:0/253817444 learned_addr learned my addr 172.21.15.37:0/253817444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f5174d8d700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 -- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 msgr2=0x55c38b0d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 -- 172.21.15.37:0/253817444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 msgr2=0x55c38b0d79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f517558e700 1 -- 172.21.15.37:0/253817444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c38b0ee750 con 0x55c38b0cbbe0 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.622+0000 7f5175d8f700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:10.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f517558e700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d6f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f516400b800 tx=0x7f516400f820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8095f81de9532077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f5158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164020040 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f516400fda0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5168080020 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5164041050 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f518207bb80 10 client.24794 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c38b14cea0 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.623+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5168080020 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5164033420 con 0x55c38b0cbbe0 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 10 client.24794 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 10 client.24794 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 20 client.24794 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:10.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b151690 0x55c38b1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 4 client.24794 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "status"}) -- 0x55c38b021570 con 0x55c38b151690 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f5174d8d700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b151690 0x55c38b1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f5158ff9700 10 client.24794 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.624+0000 7f5174d8d700 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b151690 0x55c38b1503a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e20c7461b731ed9 server_cookie=1a45b778450a0eef in_seq=0 out_seq=0 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f5158ff9700 1 -- 172.21.15.37:0/253817444 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c38b021570 con 0x55c38b151690 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f5158ff9700 10 client.24794 handle_command_reply: tid=0 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 client.24794 shutdown 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 msgr2=0x55c38b0d6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d6f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f516400b800 tx=0x7f516400f820 comp rx=0 tx=0).stop 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 shutdown_connections 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x55c38b151690 0x55c38b1503a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c38b0d0990 0x55c38b0d79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c38b0cbbe0 0x55c38b0d6f30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 --2- 172.21.15.37:0/253817444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c38b0cc500 0x55c38b0d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 >> 172.21.15.37:0/253817444 conn(0x55c38aff4640 msgr2=0x55c38b0c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f5158ff9700 0 client.24794 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 shutdown_connections 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 1 -- 172.21.15.37:0/253817444 wait complete. 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.625+0000 7f518207bb80 20 client.24794 trim_cache size 0 max 16384 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 3, 2024-04-25T21:08:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24445, 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 171.865644186, 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 223.39729120800001 2024-04-25T21:08:10.631 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 msgr2=0x7f515c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 0x7f515c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5164005fd0 tx=0x7f5164005d40 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 msgr2=0x7f517019da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 0x7f517019da00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f516800cc50 tx=0x7f516800cf60 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 shutdown_connections 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5170108a70 0x7f517019da00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5170102ac0 0x7f5170199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f515c0a3c40 0x7f515c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 --2- 172.21.15.37:0/4265635973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5170103480 0x7f5170199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 >> 172.21.15.37:0/4265635973 conn(0x7f5170079bb0 msgr2=0x7f5170101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:10.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 shutdown_connections 2024-04-25T21:08:10.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:10.626+0000 7f51777f2700 1 -- 172.21.15.37:0/4265635973 wait complete. 2024-04-25T21:08:11.334 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T21:08:11.586 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:11.963 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:11 smithi037 ceph-mon[29195]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:11 smithi177 ceph-mon[33043]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:11 smithi098 ceph-mon[32569]: pgmap v398: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 -- 172.21.15.37:0/3922583075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 msgr2=0x7f2ca8075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 --2- 172.21.15.37:0/3922583075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca8075050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009990 tx=0x7f2c98009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 -- 172.21.15.37:0/3922583075 shutdown_connections 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 --2- 172.21.15.37:0/3922583075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca8111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 --2- 172.21.15.37:0/3922583075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca8075050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 --2- 172.21.15.37:0/3922583075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 0x7f2ca810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 -- 172.21.15.37:0/3922583075 >> 172.21.15.37:0/3922583075 conn(0x7f2ca8079bb0 msgr2=0x7f2ca8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 -- 172.21.15.37:0/3922583075 shutdown_connections 2024-04-25T21:08:13.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.309+0000 7f2cae122700 1 -- 172.21.15.37:0/3922583075 wait complete. 2024-04-25T21:08:13.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 Processor -- start 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 -- start start 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 0x7f2ca819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8114ec0 con 0x7f2ca8074c70 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8114d40 con 0x7f2ca810f470 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.310+0000 7f2cae122700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8115040 con 0x7f2ca8075590 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41228/0 (socket says 172.21.15.37:41228) 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 -- 172.21.15.37:0/689438005 learned_addr learned my addr 172.21.15.37:0/689438005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 0x7f2ca819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 -- 172.21.15.37:0/689438005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 msgr2=0x7f2ca819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 0x7f2ca819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 -- 172.21.15.37:0/689438005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 msgr2=0x7f2ca819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 -- 172.21.15.37:0/689438005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81a03c0 con 0x7f2ca8074c70 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca4ff9700 1 -- 172.21.15.37:0/689438005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98027070 con 0x7f2ca8074c70 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca4ff9700 1 -- 172.21.15.37:0/689438005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980041d0 con 0x7f2ca8074c70 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81a0650 con 0x7f2ca8074c70 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2ca4ff9700 1 -- 172.21.15.37:0/689438005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800f460 con 0x7f2ca8074c70 2024-04-25T21:08:13.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.311+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a7f30 con 0x7f2ca8074c70 2024-04-25T21:08:13.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.314+0000 7f2ca4ff9700 1 -- 172.21.15.37:0/689438005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c9802a070 con 0x7f2ca8074c70 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.314+0000 7f2ca4ff9700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 0x7f2c900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.314+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 0x7f2c900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.314+0000 7f2ca4ff9700 1 -- 172.21.15.37:0/689438005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2c980d13a0 con 0x7f2ca8074c70 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 0x7f2c900a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2cb89abb80 1 Processor -- start 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2cb89abb80 1 -- start start 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d0b0 0x55d03349d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.315+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0333f0c40 con 0x55d0334a8880 2024-04-25T21:08:13.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d0333978e0 con 0x55d03349d9d0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0333ecc00 con 0x55d03349d0b0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41238/0 (socket says 172.21.15.37:41238) 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 -- 172.21.15.37:0/2815050376 learned_addr learned my addr 172.21.15.37:0/2815050376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d0b0 0x55d03349d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 -- 172.21.15.37:0/2815050376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d0b0 msgr2=0x55d03349d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d0b0 0x55d03349d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 -- 172.21.15.37:0/2815050376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 msgr2=0x55d0334a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 -- 172.21.15.37:0/2815050376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03335b6f0 con 0x55d0334a8880 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d300 tx=0x7f2ca000d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a326f156178c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2c8dffb700 1 -- 172.21.15.37:0/2815050376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0016020 con 0x55d0334a8880 2024-04-25T21:08:13.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2c8dffb700 1 -- 172.21.15.37:0/2815050376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0012aa0 con 0x55d0334a8880 2024-04-25T21:08:13.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.316+0000 7f2c8dffb700 1 -- 172.21.15.37:0/2815050376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00119a0 con 0x55d0334a8880 2024-04-25T21:08:13.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2815050376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 msgr2=0x55d0334aac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000d300 tx=0x7f2ca000d610 comp rx=0 tx=0).stop 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2815050376 shutdown_connections 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334aac70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d0b0 0x55d03349d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2815050376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2815050376 >> 172.21.15.37:0/2815050376 conn(0x55d0333ca840 msgr2=0x55d033499580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2815050376 shutdown_connections 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.317+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2815050376 wait complete. 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 -- start start 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 Processor -- start 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0334ab370 con 0x55d0334a8880 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d0333aa560 con 0x55d03349d0b0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0334ab4f0 con 0x55d03349d9d0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2cb89abb80 10 client.0 _ll_register_callbacks cb 0x55d0333898d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:49392/0 (socket says 172.21.15.37:49392) 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2ca77fe700 1 -- 172.21.15.37:0/88199073 learned_addr learned my addr 172.21.15.37:0/88199073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2ca7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.318+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 -- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 msgr2=0x55d0334a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 -- 172.21.15.37:0/88199073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 msgr2=0x55d0334a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 -- 172.21.15.37:0/88199073 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03333e880 con 0x55d03349d0b0 2024-04-25T21:08:13.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c98099b80 tx=0x7f2c98022950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d03559fb7ccef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9809d080 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca0012aa0 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2c980a6070 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c9809de20 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2cb89abb80 10 client.24800 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.319+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0334b5b10 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca0012aa0 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2c9809cb70 con 0x55d03349d0b0 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 10 client.24800 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 10 client.24800 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 20 client.24800 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:13.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d0334b8180 0x55d0334b95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 4 client.24800 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0333fe430 con 0x55d0334b8180 2024-04-25T21:08:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.320+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d0334b8180 0x55d0334b95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.321+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d0334b8180 0x55d0334b95f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4814bdae7310707a server_cookie=6259ad6151a66e2a in_seq=0 out_seq=0 2024-04-25T21:08:13.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.321+0000 7f2c86ffd700 10 client.24800 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.321+0000 7f2c86ffd700 1 -- 172.21.15.37:0/88199073 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0333fe430 con 0x55d0334b8180 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.321+0000 7f2c86ffd700 10 client.24800 handle_command_reply: tid=0 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 client.24800 shutdown 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 msgr2=0x55d0334a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c98099b80 tx=0x7f2c98022950 comp rx=0 tx=0).stop 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 shutdown_connections 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d0334a8880 0x55d0334a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d0b0 0x55d0334a7f40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d0334b8180 0x55d0334b95f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/88199073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2c86ffd700 0 client.24800 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 >> 172.21.15.37:0/88199073 conn(0x55d0333ca840 msgr2=0x55d0334991b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 shutdown_connections 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 1 -- 172.21.15.37:0/88199073 wait complete. 2024-04-25T21:08:13.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.322+0000 7f2cb89abb80 20 client.24800 trim_cache size 0 max 16384 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.342+0000 7f2cb89abb80 1 Processor -- start 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.342+0000 7f2cb89abb80 1 -- start start 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 0x55d033499040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b7930 con 0x55d03349d0b0 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b8000 con 0x55d0334a8880 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b86d0 con 0x55d03349d9d0 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41264/0 (socket says 172.21.15.37:41264) 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 -- 172.21.15.37:0/2991431303 learned_addr learned my addr 172.21.15.37:0/2991431303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 0x55d033499040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 -- 172.21.15.37:0/2991431303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 msgr2=0x55d0334a4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 -- 172.21.15.37:0/2991431303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 msgr2=0x55d033499040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 0x55d033499040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 -- 172.21.15.37:0/2991431303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0334a37b0 con 0x55d03349d0b0 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 0x55d033499040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809a0f0 tx=0x7f2c9809bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea564d42e67f60e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2c8d7fa700 1 -- 172.21.15.37:0/2991431303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98022470 con 0x55d03349d0b0 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2c8d7fa700 1 -- 172.21.15.37:0/2991431303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980bc020 con 0x55d03349d0b0 2024-04-25T21:08:13.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.343+0000 7f2c8d7fa700 1 -- 172.21.15.37:0/2991431303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c980a60f0 con 0x55d03349d0b0 2024-04-25T21:08:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2991431303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 msgr2=0x55d0334a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809a0f0 tx=0x7f2c9809bc30 comp rx=0 tx=0).stop 2024-04-25T21:08:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2991431303 shutdown_connections 2024-04-25T21:08:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d0334a8880 0x55d033499040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334a3b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/2991431303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d03349d9d0 0x55d0334a4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2991431303 >> 172.21.15.37:0/2991431303 conn(0x55d0333ca840 msgr2=0x55d0334b6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2991431303 shutdown_connections 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- 172.21.15.37:0/2991431303 wait complete. 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.344+0000 7f2cb89abb80 1 -- start start 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 Processor -- start 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 0x55d0334a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b9470 con 0x55d03349d0b0 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b8da0 con 0x55d03349d9d0 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d0334b9b40 con 0x55d0334a8880 2024-04-25T21:08:13.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2cb89abb80 10 client.0 _ll_register_callbacks cb 0x55d0333898d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41266/0 (socket says 172.21.15.37:41266) 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 -- 172.21.15.37:0/415630944 learned_addr learned my addr 172.21.15.37:0/415630944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 0x55d0334a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 -- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 msgr2=0x55d0334a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 0x55d0334a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 -- 172.21.15.37:0/415630944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 msgr2=0x55d0334a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca77fe700 1 -- 172.21.15.37:0/415630944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0334bea30 con 0x55d03349d0b0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.345+0000 7f2ca7fff700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 0x55d0334a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2ca77fe700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809b9d0 tx=0x7f2c980a40b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6648070aec6e5b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98022470 con 0x55d03349d0b0 2024-04-25T21:08:13.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f2c98003d20 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c980bc020 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2c980a1ad0 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 10 client.15123 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0334bea30 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c980bc020 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2c98083070 con 0x55d03349d0b0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 10 client.15123 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 10 client.15123 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 20 client.15123 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d033523840 0x55d033522550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 4 client.15123 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.346+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "status"}) -- 0x55d0333f84d0 con 0x55d033523840 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d033523840 0x55d033522550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:13.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2ca6ffd700 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d033523840 0x55d033522550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a7e48f347d89e4 server_cookie=aaf3b662a0c9dc84 in_seq=0 out_seq=0 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2c86ffd700 10 client.15123 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2c86ffd700 1 -- 172.21.15.37:0/415630944 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d0333f84d0 con 0x55d033523840 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2c86ffd700 10 client.15123 handle_command_reply: tid=0 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2cb89abb80 1 client.15123 shutdown 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 msgr2=0x55d0334c2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.347+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2c9809b9d0 tx=0x7f2c980a40b0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 shutdown_connections 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x55d033523840 0x55d033522550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0334a8880 0x55d0334a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d03349d0b0 0x55d0334c2f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 --2- 172.21.15.37:0/415630944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d03349d9d0 0x55d0334a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 >> 172.21.15.37:0/415630944 conn(0x55d0333ca840 msgr2=0x55d0334a3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2c86ffd700 0 client.15123 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:08:13.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 shutdown_connections 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 1 -- 172.21.15.37:0/415630944 wait complete. 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cb89abb80 20 client.15123 trim_cache size 0 max 16384 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24457, 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:13.354 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 175.60317752200001, 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 223.35505950000001 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 msgr2=0x7f2c900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 0x7f2c900a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006fd0 tx=0x7f2c9c008040 comp rx=0 tx=0).stop 2024-04-25T21:08:13.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 msgr2=0x7f2ca819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2c98000c00 tx=0x7f2c980047b0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 shutdown_connections 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f2c900a3c40 0x7f2c900a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ca810f470 0x7f2ca819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ca8074c70 0x7f2ca819b3b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 --2- 172.21.15.37:0/689438005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ca8075590 0x7f2ca819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.348+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 >> 172.21.15.37:0/689438005 conn(0x7f2ca8079bb0 msgr2=0x7f2ca8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.349+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 shutdown_connections 2024-04-25T21:08:13.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:13.349+0000 7f2cae122700 1 -- 172.21.15.37:0/689438005 wait complete. 2024-04-25T21:08:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:13 smithi037 ceph-mon[29195]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:13 smithi177 ceph-mon[33043]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:14.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T21:08:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:13 smithi098 ceph-mon[32569]: pgmap v399: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:14.354 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:15.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:15 smithi037 ceph-mon[29195]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:15.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:15 smithi098 ceph-mon[32569]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:15 smithi177 ceph-mon[33043]: pgmap v400: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- 172.21.15.37:0/588831138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350101970 msgr2=0x7f9350101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- 172.21.15.37:0/588831138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350101970 0x7f9350101df0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9340009990 tx=0x7f9340009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- 172.21.15.37:0/588831138 shutdown_connections 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- 172.21.15.37:0/588831138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350102330 0x7f935010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- 172.21.15.37:0/588831138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f9350107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- 172.21.15.37:0/588831138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350101970 0x7f9350101df0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- 172.21.15.37:0/588831138 >> 172.21.15.37:0/588831138 conn(0x7f93500788f0 msgr2=0x7f9350078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- 172.21.15.37:0/588831138 shutdown_connections 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- 172.21.15.37:0/588831138 wait complete. 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 Processor -- start 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 -- start start 2024-04-25T21:08:16.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350101970 0x7f9350199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.099+0000 7f9358343700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f935019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f9358343700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9350111b20 con 0x7f9350102330 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f9358343700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93501119a0 con 0x7f9350107920 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f9358343700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9350111ca0 con 0x7f9350101970 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41288/0 (socket says 172.21.15.37:41288) 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 -- 172.21.15.37:0/347369529 learned_addr learned my addr 172.21.15.37:0/347369529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93568e0700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f935019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93560df700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350101970 0x7f9350199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 -- 172.21.15.37:0/347369529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350101970 msgr2=0x7f9350199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350101970 0x7f9350199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 -- 172.21.15.37:0/347369529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 msgr2=0x7f935019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f935019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 -- 172.21.15.37:0/347369529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935019e120 con 0x7f9350102330 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93568e0700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f935019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93558de700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93477fe700 1 -- 172.21.15.37:0/347369529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340027070 con 0x7f9350102330 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93477fe700 1 -- 172.21.15.37:0/347369529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9340003fa0 con 0x7f9350102330 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935019e3b0 con 0x7f9350102330 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f93477fe700 1 -- 172.21.15.37:0/347369529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340022db0 con 0x7f9350102330 2024-04-25T21:08:16.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.100+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93501a5c90 con 0x7f9350102330 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.103+0000 7f93477fe700 1 -- 172.21.15.37:0/347369529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f934002a070 con 0x7f9350102330 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.103+0000 7f93477fe700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 0x7f933c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.103+0000 7f93560df700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 0x7f933c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.103+0000 7f93477fe700 1 -- 172.21.15.37:0/347369529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f93400d0fe0 con 0x7f9350102330 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f93560df700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 0x7f933c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 Processor -- start 2024-04-25T21:08:16.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 -- start start 2024-04-25T21:08:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba40b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba41d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eba35ec40 con 0x555eba40b9d0 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eba3058e0 con 0x555eba40b0b0 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.104+0000 7f9362bccb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eba35ac00 con 0x555eba41ac90 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41292/0 (socket says 172.21.15.37:41292) 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 -- 172.21.15.37:0/3875209147 learned_addr learned my addr 172.21.15.37:0/3875209147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93560df700 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba40b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93568e0700 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba41d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 -- 172.21.15.37:0/3875209147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 msgr2=0x555eba41d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba41d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 -- 172.21.15.37:0/3875209147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 msgr2=0x555eba40b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba40b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 -- 172.21.15.37:0/3875209147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eba2c96f0 con 0x555eba40b9d0 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f93558de700 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9340099dd0 tx=0x7f9340022880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5481c1c2d3e316a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f9344d70700 1 -- 172.21.15.37:0/3875209147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400a17b0 con 0x555eba40b9d0 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f9344d70700 1 -- 172.21.15.37:0/3875209147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93400a5fe0 con 0x555eba40b9d0 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.105+0000 7f9344d70700 1 -- 172.21.15.37:0/3875209147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400a5db0 con 0x555eba40b9d0 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- 172.21.15.37:0/3875209147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 msgr2=0x555eba41a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9340099dd0 tx=0x7f9340022880 comp rx=0 tx=0).stop 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- 172.21.15.37:0/3875209147 shutdown_connections 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba41d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba40b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3875209147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba41a750 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- 172.21.15.37:0/3875209147 >> 172.21.15.37:0/3875209147 conn(0x555eba338840 msgr2=0x555eba407580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- 172.21.15.37:0/3875209147 shutdown_connections 2024-04-25T21:08:16.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- 172.21.15.37:0/3875209147 wait complete. 2024-04-25T21:08:16.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.106+0000 7f9362bccb80 1 -- start start 2024-04-25T21:08:16.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 Processor -- start 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b0b0 0x555eba41a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 0x555eba417720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eba35ec40 con 0x555eba41ac90 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eba318560 con 0x555eba40b9d0 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eba41d5c0 con 0x555eba40b0b0 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9362bccb80 10 client.0 _ll_register_callbacks cb 0x555eba2f78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:16.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41302/0 (socket says 172.21.15.37:41302) 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 -- 172.21.15.37:0/3871564321 learned_addr learned my addr 172.21.15.37:0/3871564321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93560df700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b0b0 0x555eba41a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93558de700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 0x555eba417720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 -- 172.21.15.37:0/3871564321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b0b0 msgr2=0x555eba41a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b0b0 0x555eba41a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 -- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 msgr2=0x555eba417720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 0x555eba417720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 -- 172.21.15.37:0/3871564321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eba2ac880 con 0x555eba41ac90 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93558de700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 0x555eba417720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f93568e0700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9348007ba0 tx=0x7f9348007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d19f362310c217a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.107+0000 7f9339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348010830 con 0x555eba41ac90 2024-04-25T21:08:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f93480109d0 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93400a5fe0 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9348010bd0 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 10 client.15135 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555eba427d80 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93400a5fe0 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f934801f490 con 0x555eba41ac90 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 10 client.15135 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 20 client.15135 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba42a430 0x555eba42b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 4 client.15135 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.108+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eba36c430 con 0x555eba42a430 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.109+0000 7f93560df700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba42a430 0x555eba42b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.109+0000 7f93560df700 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba42a430 0x555eba42b8a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d044600330ff411 server_cookie=3969665907d7deea in_seq=0 out_seq=0 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.109+0000 7f9339ffb700 10 client.15135 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9339ffb700 1 -- 172.21.15.37:0/3871564321 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555eba36c430 con 0x555eba42a430 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9339ffb700 10 client.15135 handle_command_reply: tid=0 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 client.15135 shutdown 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 msgr2=0x555eba417c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9348007ba0 tx=0x7f9348007eb0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 shutdown_connections 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba42a430 0x555eba42b8a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba417c60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b0b0 0x555eba41a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 --2- 172.21.15.37:0/3871564321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b9d0 0x555eba417720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9339ffb700 0 client.15135 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 >> 172.21.15.37:0/3871564321 conn(0x555eba338840 msgr2=0x555eba4071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 shutdown_connections 2024-04-25T21:08:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 1 -- 172.21.15.37:0/3871564321 wait complete. 2024-04-25T21:08:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.110+0000 7f9362bccb80 20 client.15135 trim_cache size 0 max 16384 2024-04-25T21:08:16.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 Processor -- start 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 -- start start 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba417cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 0x555eba434d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eba429be0 con 0x555eba41ac90 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eba42a2b0 con 0x555eba40b0b0 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f9362bccb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eba42a980 con 0x555eba40b9d0 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f93568e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f93568e0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41314/0 (socket says 172.21.15.37:41314) 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.131+0000 7f93568e0700 1 -- 172.21.15.37:0/312628009 learned_addr learned my addr 172.21.15.37:0/312628009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93560df700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba417cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93558de700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 0x555eba434d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 -- 172.21.15.37:0/312628009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 msgr2=0x555eba434d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 0x555eba434d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 -- 172.21.15.37:0/312628009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 msgr2=0x555eba417cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba417cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 -- 172.21.15.37:0/312628009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eba4082a0 con 0x555eba41ac90 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93558de700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 0x555eba434d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f93568e0700 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9348007fd0 tx=0x7f9348005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2439af897d80c2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f933bfff700 1 -- 172.21.15.37:0/312628009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348010d70 con 0x555eba41ac90 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f933bfff700 1 -- 172.21.15.37:0/312628009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934801b070 con 0x555eba41ac90 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.132+0000 7f933bfff700 1 -- 172.21.15.37:0/312628009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348005c60 con 0x555eba41ac90 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- 172.21.15.37:0/312628009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 msgr2=0x555eba435240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9348007fd0 tx=0x7f9348005b00 comp rx=0 tx=0).stop 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- 172.21.15.37:0/312628009 shutdown_connections 2024-04-25T21:08:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba41ac90 0x555eba435240 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba417cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- 172.21.15.37:0/312628009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba40b9d0 0x555eba434d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- 172.21.15.37:0/312628009 >> 172.21.15.37:0/312628009 conn(0x555eba338840 msgr2=0x555eba407060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- 172.21.15.37:0/312628009 shutdown_connections 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- 172.21.15.37:0/312628009 wait complete. 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- start start 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 Processor -- start 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba414a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba433320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eba42b720 con 0x555eba40b9d0 2024-04-25T21:08:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eba42b050 con 0x555eba40b0b0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eba42bdf0 con 0x555eba41ac90 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.133+0000 7f9362bccb80 10 client.0 _ll_register_callbacks cb 0x555eba2f78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41326/0 (socket says 172.21.15.37:41326) 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93560df700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba414a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 -- 172.21.15.37:0/1432777143 learned_addr learned my addr 172.21.15.37:0/1432777143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93568e0700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba433320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 -- 172.21.15.37:0/1432777143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 msgr2=0x555eba433320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba433320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 -- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 msgr2=0x555eba414a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba414a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 -- 172.21.15.37:0/1432777143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eba42c800 con 0x555eba40b9d0 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93568e0700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba433320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:16.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f93558de700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f934009d340 tx=0x7f9340005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d274e8a6a8ee946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9339ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400a17b0 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f9340004150 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f934801b070 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93400a5fe0 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9362bccb80 10 client.15141 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.134+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555eba491a60 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934801b070 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f934009c850 con 0x555eba40b9d0 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 20 client.15141 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba496390 0x555eba4950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:16.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 4 client.15141 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:16.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "status"}) -- 0x555eba3664d0 con 0x555eba496390 2024-04-25T21:08:16.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.135+0000 7f93560df700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba496390 0x555eba4950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:16.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f93560df700 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba496390 0x555eba4950a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11f1fe2c8504926c server_cookie=90d3a940accd4e3e in_seq=0 out_seq=0 2024-04-25T21:08:16.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9339ffb700 10 client.15141 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9339ffb700 1 -- 172.21.15.37:0/1432777143 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555eba3664d0 con 0x555eba496390 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9339ffb700 10 client.15141 handle_command_reply: tid=0 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 client.15141 shutdown 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 msgr2=0x555eba414fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f934009d340 tx=0x7f9340005c00 comp rx=0 tx=0).stop 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 shutdown_connections 2024-04-25T21:08:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eba41ac90 0x555eba433320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eba40b0b0 0x555eba414a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x555eba496390 0x555eba4950a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 --2- 172.21.15.37:0/1432777143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eba40b9d0 0x555eba414fc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9339ffb700 0 client.15141 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 >> 172.21.15.37:0/1432777143 conn(0x555eba338840 msgr2=0x555eba415ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 shutdown_connections 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 1 -- 172.21.15.37:0/1432777143 wait complete. 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.136+0000 7f9362bccb80 20 client.15141 trim_cache size 0 max 16384 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:16.143 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 179.397016745, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 209.90875593000001 2024-04-25T21:08:16.144 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 msgr2=0x7f933c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 0x7f933c0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c005ce0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 msgr2=0x7f93501996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9340000c00 tx=0x7f93400047b0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 shutdown_connections 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f933c0a3cc0 0x7f933c0a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350107920 0x7f935019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9350101970 0x7f9350199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 --2- 172.21.15.37:0/347369529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9350102330 0x7f93501996a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 >> 172.21.15.37:0/347369529 conn(0x7f93500788f0 msgr2=0x7f935006b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 shutdown_connections 2024-04-25T21:08:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:16.137+0000 7f9358343700 1 -- 172.21.15.37:0/347369529 wait complete. 2024-04-25T21:08:16.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T21:08:17.195 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:17 smithi037 ceph-mon[29195]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:17 smithi177 ceph-mon[33043]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:17 smithi098 ceph-mon[32569]: pgmap v401: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 -- 172.21.15.37:0/506548561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8107a60 msgr2=0x7f49a8107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 --2- 172.21.15.37:0/506548561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8107a60 0x7f49a8107e40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f49980099c0 tx=0x7f4998009cd0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 -- 172.21.15.37:0/506548561 shutdown_connections 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 --2- 172.21.15.37:0/506548561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 --2- 172.21.15.37:0/506548561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8107a60 0x7f49a8107e40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 --2- 172.21.15.37:0/506548561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8100f90 0x7f49a8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 -- 172.21.15.37:0/506548561 >> 172.21.15.37:0/506548561 conn(0x7f49a8077f40 msgr2=0x7f49a80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 -- 172.21.15.37:0/506548561 shutdown_connections 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.893+0000 7f49ae08f700 1 -- 172.21.15.37:0/506548561 wait complete. 2024-04-25T21:08:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 Processor -- start 2024-04-25T21:08:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 -- start start 2024-04-25T21:08:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 0x7f49a819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112d70 con 0x7f49a8100f90 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112bf0 con 0x7f49a8101950 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49ae08f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f49a8112ef0 con 0x7f49a8107a60 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60100/0 (socket says 172.21.15.37:60100) 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 -- 172.21.15.37:0/377525592 learned_addr learned my addr 172.21.15.37:0/377525592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a7fff700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 0x7f49a819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 -- 172.21.15.37:0/377525592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 msgr2=0x7f49a819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 0x7f49a819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 -- 172.21.15.37:0/377525592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 msgr2=0x7f49a819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a77fe700 1 -- 172.21.15.37:0/377525592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81a0350 con 0x7f49a8100f90 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a7fff700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 0x7f49a819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.894+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49a77fe700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49980098e0 tx=0x7f4998003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49a4ff9700 1 -- 172.21.15.37:0/377525592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998027070 con 0x7f49a8100f90 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49a4ff9700 1 -- 172.21.15.37:0/377525592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49980040a0 con 0x7f49a8100f90 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a81a05e0 con 0x7f49a8100f90 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49a4ff9700 1 -- 172.21.15.37:0/377525592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499800f460 con 0x7f49a8100f90 2024-04-25T21:08:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.895+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a7ec0 con 0x7f49a8100f90 2024-04-25T21:08:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.897+0000 7f49a4ff9700 1 -- 172.21.15.37:0/377525592 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f499802a070 con 0x7f49a8100f90 2024-04-25T21:08:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.898+0000 7f49a4ff9700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 0x7f49900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.898+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 0x7f49900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.898+0000 7f49a4ff9700 1 -- 172.21.15.37:0/377525592 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f49980d1470 con 0x7f49a8100f90 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.898+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 0x7f49900a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f499c005fd0 tx=0x7f499c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 Processor -- start 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 -- start start 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 0x560a42747bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a4274c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a42699ce0 con 0x560a427480f0 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a42640910 con 0x560a4274c580 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49b8918b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a42695ca0 con 0x560a427477d0 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60104/0 (socket says 172.21.15.37:60104) 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3838955538 learned_addr learned my addr 172.21.15.37:0/3838955538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a77fe700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 0x560a42747bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a7fff700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a4274c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3838955538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 msgr2=0x560a42747bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 0x560a42747bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3838955538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 msgr2=0x560a4274c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a4274c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3838955538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a425e7880 con 0x560a427480f0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.899+0000 7f49a77fe700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 0x560a42747bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f499c00dc30 tx=0x7f499c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a623f2306ceb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f498dffb700 1 -- 172.21.15.37:0/3838955538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c020070 con 0x560a427480f0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f498dffb700 1 -- 172.21.15.37:0/3838955538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c01c450 con 0x560a427480f0 2024-04-25T21:08:18.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f498dffb700 1 -- 172.21.15.37:0/3838955538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00fe60 con 0x560a427480f0 2024-04-25T21:08:18.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f49b8918b80 1 -- 172.21.15.37:0/3838955538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 msgr2=0x560a4274c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.900+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f499c00dc30 tx=0x7f499c010000 comp rx=0 tx=0).stop 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- 172.21.15.37:0/3838955538 shutdown_connections 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a4274c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427477d0 0x560a42747bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3838955538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274c040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- 172.21.15.37:0/3838955538 >> 172.21.15.37:0/3838955538 conn(0x560a42674640 msgr2=0x560a42742820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- 172.21.15.37:0/3838955538 shutdown_connections 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- 172.21.15.37:0/3838955538 wait complete. 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- start start 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 Processor -- start 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42750d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42751290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a42754610 con 0x560a427477d0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a42638b80 con 0x560a4274c580 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.901+0000 7f49b8918b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a42754790 con 0x560a427480f0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49b8918b80 10 client.0 _ll_register_callbacks cb 0x560a42632900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60120/0 (socket says 172.21.15.37:60120) 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 -- 172.21.15.37:0/1822821261 learned_addr learned my addr 172.21.15.37:0/1822821261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42750d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 -- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 msgr2=0x560a42750d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42750d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a7fff700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42751290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 -- 172.21.15.37:0/1822821261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 msgr2=0x560a42751290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42751290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 -- 172.21.15.37:0/1822821261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a42651220 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42750d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49a77fe700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4998009990 tx=0x7f4998009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788e31d79ca6ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f4986ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499809de90 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f499809d9f0 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499c01c450 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f499809d790 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49b8918b80 10 client.15153 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.902+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a4275eea0 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c01c450 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f49980b6060 con 0x560a427477d0 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 10 client.15153 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:08:18.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 20 client.15153 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a42761560 0x560a427629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 4 client.15153 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.903+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a426a74d0 con 0x560a42761560 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.904+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a42761560 0x560a427629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.904+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a42761560 0x560a427629d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d454275670780cca server_cookie=a6380f03bbe5d933 in_seq=0 out_seq=0 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.904+0000 7f4986ffd700 10 client.15153 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f4986ffd700 1 -- 172.21.15.37:0/1822821261 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a426a74d0 con 0x560a42761560 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f4986ffd700 10 client.15153 handle_command_reply: tid=0 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 client.15153 shutdown 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 msgr2=0x560a42753a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4998009990 tx=0x7f4998009f10 comp rx=0 tx=0).stop 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 shutdown_connections 2024-04-25T21:08:18.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a42761560 0x560a427629d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42751290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f4986ffd700 0 client.15153 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a42753a50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 --2- 172.21.15.37:0/1822821261 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42750d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 >> 172.21.15.37:0/1822821261 conn(0x560a42674640 msgr2=0x560a4274d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 shutdown_connections 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 1 -- 172.21.15.37:0/1822821261 wait complete. 2024-04-25T21:08:18.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.905+0000 7f49b8918b80 20 client.15153 trim_cache size 0 max 16384 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 Processor -- start 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 -- start start 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 0x560a42752c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 0x560a4274e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a42760d10 con 0x560a427480f0 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a427613e0 con 0x560a427477d0 2024-04-25T21:08:18.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49b8918b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a42761ab0 con 0x560a4274c580 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60126/0 (socket says 172.21.15.37:60126) 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3577296588 learned_addr learned my addr 172.21.15.37:0/3577296588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a7fff700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 0x560a4274e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3577296588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 msgr2=0x560a4274e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a77fe700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 0x560a42752c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 0x560a4274e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3577296588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 msgr2=0x560a42752c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 0x560a42752c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.926+0000 7f49a6ffd700 1 -- 172.21.15.37:0/3577296588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4274f2e0 con 0x560a427480f0 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f49a77fe700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 0x560a42752c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f49a7fff700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 0x560a4274e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f499c008fd0 tx=0x7f499c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e571f3f1a378c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f498d7fa700 1 -- 172.21.15.37:0/3577296588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c037070 con 0x560a427480f0 2024-04-25T21:08:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f498d7fa700 1 -- 172.21.15.37:0/3577296588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c01cbd0 con 0x560a427480f0 2024-04-25T21:08:18.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.927+0000 7f498d7fa700 1 -- 172.21.15.37:0/3577296588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c013040 con 0x560a427480f0 2024-04-25T21:08:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- 172.21.15.37:0/3577296588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 msgr2=0x560a4274e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f499c008fd0 tx=0x7f499c00dbb0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- 172.21.15.37:0/3577296588 shutdown_connections 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a4274c580 0x560a4274e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a427477d0 0x560a42752c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 --2- 172.21.15.37:0/3577296588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427480f0 0x560a4274e060 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- 172.21.15.37:0/3577296588 >> 172.21.15.37:0/3577296588 conn(0x560a42674640 msgr2=0x560a42752580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- 172.21.15.37:0/3577296588 shutdown_connections 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- 172.21.15.37:0/3577296588 wait complete. 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.928+0000 7f49b8918b80 1 -- start start 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 Processor -- start 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42766420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42766960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a42762850 con 0x560a427477d0 2024-04-25T21:08:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a42762180 con 0x560a4274c580 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a42762f20 con 0x560a427480f0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49b8918b80 10 client.0 _ll_register_callbacks cb 0x560a42632900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60140/0 (socket says 172.21.15.37:60140) 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 -- 172.21.15.37:0/4167217248 learned_addr learned my addr 172.21.15.37:0/4167217248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a7fff700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42766960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 -- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 msgr2=0x560a42766420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42766420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 -- 172.21.15.37:0/4167217248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 msgr2=0x560a42766960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42766960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a77fe700 1 -- 172.21.15.37:0/4167217248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a427670b0 con 0x560a427477d0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.929+0000 7f49a7fff700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42766960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f49a77fe700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f499800c010 tx=0x7f49980054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b8787d4cfc7bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499809d2d0 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4998003e00 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499c01cbd0 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f499809de90 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f49b8918b80 10 client.15159 fetch_fsmap learned FSMap version 26 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a427670b0 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c01cbd0 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f499809eac0 con 0x560a427477d0 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.930+0000 7f49b8918b80 10 client.15159 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49b8918b80 10 client.15159 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49b8918b80 20 client.15159 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a427cde80 0x560a427ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49b8918b80 4 client.15159 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "status"}) -- 0x560a426a1570 con 0x560a427cde80 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a427cde80 0x560a427ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f49a6ffd700 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a427cde80 0x560a427ccb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae568a08ada8fff9 server_cookie=1b92d73b9c964948 in_seq=0 out_seq=0 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.931+0000 7f4986ffd700 10 client.15159 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f4986ffd700 1 -- 172.21.15.37:0/4167217248 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560a426a1570 con 0x560a427cde80 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f4986ffd700 10 client.15159 handle_command_reply: tid=0 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 client.15159 shutdown 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 msgr2=0x560a4276c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f499800c010 tx=0x7f49980054f0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 shutdown_connections 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a427cde80 0x560a427ccb90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a4274c580 0x560a42766960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a427477d0 0x560a4276c0b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 --2- 172.21.15.37:0/4167217248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a427480f0 0x560a42766420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 >> 172.21.15.37:0/4167217248 conn(0x560a42674640 msgr2=0x560a42746e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f4986ffd700 0 client.15159 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 shutdown_connections 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 1 -- 172.21.15.37:0/4167217248 wait complete. 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.932+0000 7f49b8918b80 20 client.15159 trim_cache size 0 max 16384 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24469, 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:18.937 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 194.269923621, 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 26, 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 90, 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 88, 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 222.88346674799999 2024-04-25T21:08:18.938 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 msgr2=0x7f49900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 0x7f49900a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f499c005fd0 tx=0x7f499c005e20 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 msgr2=0x7f49a819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49980098e0 tx=0x7f4998003680 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 shutdown_connections 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49a8107a60 0x7f49a819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49a8100f90 0x7f49a819b340 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f49900a3cc0 0x7f49900a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 --2- 172.21.15.37:0/377525592 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49a8101950 0x7f49a819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 >> 172.21.15.37:0/377525592 conn(0x7f49a8077f40 msgr2=0x7f49a8105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 shutdown_connections 2024-04-25T21:08:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:18.933+0000 7f49ae08f700 1 -- 172.21.15.37:0/377525592 wait complete. 2024-04-25T21:08:19.772 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T21:08:19.787 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T21:08:19.788 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T21:08:19.788 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T21:08:19.788 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T21:08:19.789 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:08:19.789 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:08:19.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T21:08:19.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 -- 172.21.15.37:0/2042616739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074100e60 msgr2=0x7f0074101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:19.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 --2- 172.21.15.37:0/2042616739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074100e60 0x7f0074101280 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f006c009900 tx=0x7f006c02a410 comp rx=0 tx=0).stop 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 -- 172.21.15.37:0/2042616739 shutdown_connections 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 --2- 172.21.15.37:0/2042616739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102a00 0x7f0074109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 --2- 172.21.15.37:0/2042616739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074102040 0x7f00741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.898+0000 7f00791bf640 1 --2- 172.21.15.37:0/2042616739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074100e60 0x7f0074101280 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.899+0000 7f00791bf640 1 -- 172.21.15.37:0/2042616739 >> 172.21.15.37:0/2042616739 conn(0x7f00740fc420 msgr2=0x7f00740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.899+0000 7f00791bf640 1 -- 172.21.15.37:0/2042616739 shutdown_connections 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.899+0000 7f00791bf640 1 -- 172.21.15.37:0/2042616739 wait complete. 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.899+0000 7f00791bf640 1 Processor -- start 2024-04-25T21:08:19.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.899+0000 7f00791bf640 1 -- start start 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074100e60 0x7f0074198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102040 0x7f0074198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f007410ca70 con 0x7f0074102a00 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f007410c8f0 con 0x7f0074102040 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f007410cbf0 con 0x7f0074100e60 2024-04-25T21:08:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60156/0 (socket says 172.21.15.37:60156) 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 -- 172.21.15.37:0/4040285509 learned_addr learned my addr 172.21.15.37:0/4040285509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007259c640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102040 0x7f0074198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f0072d9d640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074100e60 0x7f0074198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 -- 172.21.15.37:0/4040285509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074100e60 msgr2=0x7f0074198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074100e60 0x7f0074198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 -- 172.21.15.37:0/4040285509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102040 msgr2=0x7f0074198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102040 0x7f0074198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 -- 172.21.15.37:0/4040285509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007419fe20 con 0x7f0074102a00 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f007359e640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f006400b970 tx=0x7f006400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0064019070 con 0x7f0074102a00 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0064014440 con 0x7f0074102a00 2024-04-25T21:08:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00741a0110 con 0x7f0074102a00 2024-04-25T21:08:19.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.900+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640075d0 con 0x7f0074102a00 2024-04-25T21:08:19.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.901+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00741a0690 con 0x7f0074102a00 2024-04-25T21:08:19.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.901+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00741a0950 con 0x7f0074102a00 2024-04-25T21:08:19.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.903+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f006401c070 con 0x7f0074102a00 2024-04-25T21:08:19.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.903+0000 7f006bfff640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 0x7f00600a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:19.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.903+0000 7f0072d9d640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 0x7f00600a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:19.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.903+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f00640cfa20 con 0x7f0074102a00 2024-04-25T21:08:19.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.904+0000 7f0072d9d640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 0x7f00600a6110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:19.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:19.908+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f006409c410 con 0x7f0074102a00 2024-04-25T21:08:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:19 smithi037 ceph-mon[29195]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:19 smithi177 ceph-mon[33043]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:20.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.145+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f00741a0b20 con 0x7f0074102a00 2024-04-25T21:08:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:19 smithi098 ceph-mon[32569]: pgmap v402: 97 pgs: 97 active+clean; 643 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:08:20.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.700+0000 7f006bfff640 1 -- 172.21.15.37:0/4040285509 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f006409bd10 con 0x7f0074102a00 2024-04-25T21:08:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.702+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 msgr2=0x7f00600a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.702+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 0x7f00600a6110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c0314a0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.702+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 msgr2=0x7f007419f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:20.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.702+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f006400b970 tx=0x7f006400be30 comp rx=0 tx=0).stop 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 shutdown_connections 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f00600a3c50 0x7f00600a6110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0074102a00 0x7f007419f7b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0074102040 0x7f0074198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 --2- 172.21.15.37:0/4040285509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0074100e60 0x7f0074198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.703+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 >> 172.21.15.37:0/4040285509 conn(0x7f00740fc420 msgr2=0x7f00740fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:20.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.704+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 shutdown_connections 2024-04-25T21:08:20.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.705+0000 7f00791bf640 1 -- 172.21.15.37:0/4040285509 wait complete. 2024-04-25T21:08:20.721 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 -- 172.21.15.37:0/3486412332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 msgr2=0x7f537410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 --2- 172.21.15.37:0/3486412332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f537410a140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f536c009ca0 tx=0x7f536c02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 -- 172.21.15.37:0/3486412332 shutdown_connections 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 --2- 172.21.15.37:0/3486412332 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f537410a140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 --2- 172.21.15.37:0/3486412332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374105420 0x7f5374107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 --2- 172.21.15.37:0/3486412332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374102af0 0x7f5374104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 -- 172.21.15.37:0/3486412332 >> 172.21.15.37:0/3486412332 conn(0x7f53740fc420 msgr2=0x7f53740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 -- 172.21.15.37:0/3486412332 shutdown_connections 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.831+0000 7f5379129640 1 -- 172.21.15.37:0/3486412332 wait complete. 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 Processor -- start 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 -- start start 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374102af0 0x7f5374102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f537410db00 con 0x7f5374105420 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d980 con 0x7f5374107d50 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.832+0000 7f5379129640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f537410dc80 con 0x7f5374102af0 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60198/0 (socket says 172.21.15.37:60198) 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374102af0 0x7f5374102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 -- 172.21.15.37:0/1963630088 learned_addr learned my addr 172.21.15.37:0/1963630088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:36694/0 (socket says 172.21.15.37:36694) 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374102af0 msgr2=0x7f5374102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374102af0 0x7f5374102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 msgr2=0x7f5374100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 -- 172.21.15.37:0/1963630088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5374101870 con 0x7f5374105420 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537359e640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f537259c640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400ccc0 con 0x7f5374105420 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a2270 con 0x7f5374105420 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53640054d0 con 0x7f5374105420 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a27c0 con 0x7f5374105420 2024-04-25T21:08:20.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.833+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364010430 con 0x7f5374105420 2024-04-25T21:08:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.834+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53740fdf10 con 0x7f5374105420 2024-04-25T21:08:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.837+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f53640046c0 con 0x7f5374105420 2024-04-25T21:08:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.837+0000 7f535bfff640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 0x7f535c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.837+0000 7f5372d9d640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 msgr2=0x7f535c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.838+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f53640d0a30 con 0x7f5374105420 2024-04-25T21:08:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.838+0000 7f5372d9d640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 0x7f535c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T21:08:20.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:20.841+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5364053ff0 con 0x7f5374105420 2024-04-25T21:08:20.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:20 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T21:08:20.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:20 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:20.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:20 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:20.985 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:20 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:20.893+0000 7f681555e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T21:08:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.031+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f537402d5c0 con 0x7f5374105420 2024-04-25T21:08:21.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.037+0000 7f535bfff640 1 -- 172.21.15.37:0/1963630088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f536409d420 con 0x7f5374105420 2024-04-25T21:08:21.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5372d9d640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 msgr2=0x7f535c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:21.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5372d9d640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 0x7f535c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 msgr2=0x7f535c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 0x7f535c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 msgr2=0x7f5374100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.039+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400eeb0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 shutdown_connections 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f535c0a3ca0 0x7f535c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5374107d50 0x7f5374100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5374105420 0x7f5374100aa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 --2- 172.21.15.37:0/1963630088 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5374102af0 0x7f5374102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 >> 172.21.15.37:0/1963630088 conn(0x7f53740fc420 msgr2=0x7f5374103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:21.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 shutdown_connections 2024-04-25T21:08:21.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.040+0000 7f5379129640 1 -- 172.21.15.37:0/1963630088 wait complete. 2024-04-25T21:08:21.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T21:08:21.064 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:20 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T21:08:21.064 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:20 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:21.064 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:20 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:21.064 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:20 smithi098 ceph-mgr[33966]: -- 172.21.15.98:0/3099387339 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604ab42a5a0 con 0x5604ab3fe800 2024-04-25T21:08:21.064 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:20 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:20.881+0000 7efd7536c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T21:08:21.067 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:20 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T21:08:21.067 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:21.067 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:21.067 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:20 smithi177 ceph-mgr[34897]: -- 172.21.15.177:0/4232673085 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c60501e5a0 con 0x55c604ff2800 2024-04-25T21:08:21.068 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:20.883+0000 7f9936bdc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T21:08:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.156+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/3325497172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68101200 msgr2=0x7f4a68101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.156+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/3325497172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68101200 0x7f4a68101660 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009900 tx=0x7f4a5c02a480 comp rx=0 tx=0).stop 2024-04-25T21:08:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/3325497172 shutdown_connections 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/3325497172 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a68102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/3325497172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a68101200 0x7f4a68101660 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/3325497172 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a681008a0 0x7f4a68100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/3325497172 >> 172.21.15.37:0/3325497172 conn(0x7f4a680fc420 msgr2=0x7f4a680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/3325497172 shutdown_connections 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/3325497172 wait complete. 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 Processor -- start 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 -- start start 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.157+0000 7f4a6d7f7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 0x7f4a6819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a6d7f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a681a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a6d7f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810fcf0 con 0x7f4a681008a0 2024-04-25T21:08:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a6d7f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810fb70 con 0x7f4a68101ba0 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a6d7f7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6810fe70 con 0x7f4a68101200 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60208/0 (socket says 172.21.15.37:60208) 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 -- 172.21.15.37:0/1184720802 learned_addr learned my addr 172.21.15.37:0/1184720802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a667fc640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 0x7f4a6819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a677fe640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a681a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 msgr2=0x7f4a6819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 0x7f4a6819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 msgr2=0x7f4a681a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a681a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 -- 172.21.15.37:0/1184720802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a681a1f70 con 0x7f4a681008a0 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a677fe640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a681a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a667fc640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 0x7f4a6819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a66ffd640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800e9f0 tx=0x7f4a5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.158+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800cd80 con 0x7f4a681008a0 2024-04-25T21:08:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.159+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a580054d0 con 0x7f4a681008a0 2024-04-25T21:08:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.159+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a681a2260 con 0x7f4a681008a0 2024-04-25T21:08:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.159+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58010690 con 0x7f4a681008a0 2024-04-25T21:08:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.159+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a2730 con 0x7f4a681008a0 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.160+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a6810b5f0 con 0x7f4a681008a0 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.161+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4a58004e90 con 0x7f4a681008a0 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.162+0000 7f4a4ffff640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 0x7f4a500a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.162+0000 7f4a667fc640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 msgr2=0x7f4a500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.162+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f4a580d0840 con 0x7f4a681008a0 2024-04-25T21:08:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.162+0000 7f4a667fc640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 0x7f4a500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T21:08:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.167+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a5809d230 con 0x7f4a681008a0 2024-04-25T21:08:21.331 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:21.090+0000 7f681555e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:21.331 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:21.204+0000 7f681555e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T21:08:21.339 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:21.067+0000 7f9936bdc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:21.339 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:21.171+0000 7f9936bdc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T21:08:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.361+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4a6802d5c0 con 0x7f4a681008a0 2024-04-25T21:08:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.363+0000 7f4a667fc640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 msgr2=0x7f4a500a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.363+0000 7f4a667fc640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 0x7f4a500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T21:08:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.366+0000 7f4a4ffff640 1 -- 172.21.15.37:0/1184720802 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f4a6802d5c0 con 0x7f4a681008a0 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 msgr2=0x7f4a500a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 0x7f4a500a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 msgr2=0x7f4a6819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800e9f0 tx=0x7f4a5800edf0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 shutdown_connections 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f4a500a3ca0 0x7f4a500a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a68101ba0 0x7f4a681a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a68101200 0x7f4a6819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 --2- 172.21.15.37:0/1184720802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a681008a0 0x7f4a6819a350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 >> 172.21.15.37:0/1184720802 conn(0x7f4a680fc420 msgr2=0x7f4a680ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 shutdown_connections 2024-04-25T21:08:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:21.369+0000 7f4a6d7f7640 1 -- 172.21.15.37:0/1184720802 wait complete. 2024-04-25T21:08:21.380 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T21:08:21.404 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T21:08:21.404 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T21:08:21.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:21.404 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T21:08:21.417 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:21.064+0000 7efd7536c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:21.417 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:21.166+0000 7efd7536c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T21:08:21.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:21.421 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T21:08:21.487 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-25T21:08:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:08:21.489 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T21:08:21.490 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T21:08:21.490 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:21.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T21:08:21.648 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:21.554+0000 7f681555e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T21:08:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T21:08:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-mon[33043]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T21:08:21.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1963630088' entity='client.admin' 2024-04-25T21:08:21.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1184720802' entity='client.admin' 2024-04-25T21:08:21.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:21.478+0000 7f9936bdc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T21:08:21.840 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:21.563+0000 7f9936bdc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:21.475+0000 7efd7536c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:21.560+0000 7efd7536c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-mon[32569]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1963630088' entity='client.admin' 2024-04-25T21:08:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:21 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1184720802' entity='client.admin' 2024-04-25T21:08:21.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4040285509' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T21:08:21.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-mon[29195]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-04-25T21:08:21.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1963630088' entity='client.admin' 2024-04-25T21:08:21.927 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1184720802' entity='client.admin' 2024-04-25T21:08:21.927 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:21.647+0000 7f681555e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T21:08:23.079 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:22 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:22.740+0000 7efd7536c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T21:08:23.080 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:22 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:22.911+0000 7efd7536c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:23.084 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:22.744+0000 7f9936bdc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T21:08:23.084 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:22.914+0000 7f9936bdc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:23.085 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:23.084+0000 7f9936bdc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T21:08:23.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:22 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:22.945+0000 7f681555e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T21:08:23.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:23.127+0000 7f681555e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:23.365 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:23.080+0000 7efd7536c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- 172.21.15.37:0/2628004746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 msgr2=0x7fc31006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 --2- 172.21.15.37:0/2628004746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc31006abb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- 172.21.15.37:0/2628004746 shutdown_connections 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 --2- 172.21.15.37:0/2628004746 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 0x7fc31010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 --2- 172.21.15.37:0/2628004746 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 0x7fc310108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 --2- 172.21.15.37:0/2628004746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc31006abb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- 172.21.15.37:0/2628004746 >> 172.21.15.37:0/2628004746 conn(0x7fc3100788f0 msgr2=0x7fc310078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- 172.21.15.37:0/2628004746 shutdown_connections 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- 172.21.15.37:0/2628004746 wait complete. 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 Processor -- start 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.541+0000 7fc317728700 1 -- start start 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 0x7fc310199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 0x7fc31019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc310112bd0 con 0x7fc31006a750 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc310112a50 con 0x7fc31006b0f0 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc317728700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc310112d50 con 0x7fc310108a30 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60232/0 (socket says 172.21.15.37:60232) 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 -- 172.21.15.37:0/1212448855 learned_addr learned my addr 172.21.15.37:0/1212448855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc314cc3700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 0x7fc310199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc315cc5700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 0x7fc31019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 msgr2=0x7fc31019da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 0x7fc31019da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 msgr2=0x7fc310199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 0x7fc310199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 -- 172.21.15.37:0/1212448855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31019e150 con 0x7fc31006a750 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.542+0000 7fc3154c4700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00d980 tx=0x7fc30c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.543+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c017070 con 0x7fc31006a750 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.543+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31019e440 con 0x7fc31006a750 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.543+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30c005020 con 0x7fc31006a750 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.543+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00fa10 con 0x7fc31006a750 2024-04-25T21:08:23.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.543+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3101a5cc0 con 0x7fc31006a750 2024-04-25T21:08:23.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.544+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc310069730 con 0x7fc31006a750 2024-04-25T21:08:23.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.545+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fc30c004a30 con 0x7fc31006a750 2024-04-25T21:08:23.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.545+0000 7fc3067fc700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 0x7fc2fc0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:23.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.545+0000 7fc314cc3700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 msgr2=0x7fc2fc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:23.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.546+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7fc30c0d0240 con 0x7fc31006a750 2024-04-25T21:08:23.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.546+0000 7fc314cc3700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 0x7fc2fc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T21:08:23.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.550+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc30c020020 con 0x7fc31006a750 2024-04-25T21:08:23.632 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:23.306+0000 7f681555e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T21:08:23.659 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:23.366+0000 7efd7536c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T21:08:23.659 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:23.458+0000 7efd7536c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T21:08:23.667 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:23.371+0000 7f9936bdc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T21:08:23.667 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:23.463+0000 7f9936bdc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T21:08:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.747+0000 7fc314cc3700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 msgr2=0x7fc2fc0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:23.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.747+0000 7fc314cc3700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 0x7fc2fc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T21:08:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.827+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc31002df80 con 0x7fc31006a750 2024-04-25T21:08:23.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.827+0000 7fc3067fc700 1 -- 172.21.15.37:0/1212448855 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fc30c09cc30 con 0x7fc31006a750 2024-04-25T21:08:23.828 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T21:08:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 msgr2=0x7fc2fc0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:08:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 0x7fc2fc0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 msgr2=0x7fc310199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00d980 tx=0x7fc30c00dc90 comp rx=0 tx=0).stop 2024-04-25T21:08:23.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 shutdown_connections 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc310108a30 0x7fc31019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc31006a750 0x7fc310199140 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7fc2fc0a3c90 0x7fc2fc0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 --2- 172.21.15.37:0/1212448855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc31006b0f0 0x7fc310199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 >> 172.21.15.37:0/1212448855 conn(0x7fc3100788f0 msgr2=0x7fc3100ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 shutdown_connections 2024-04-25T21:08:23.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:23.830+0000 7fc317728700 1 -- 172.21.15.37:0/1212448855 wait complete. 2024-04-25T21:08:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1212448855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T21:08:23.917 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:23.660+0000 7efd7536c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T21:08:23.918 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:23.853+0000 7efd7536c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:23.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1212448855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T21:08:23.945 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:23.632+0000 7f681555e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T21:08:23.945 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:23.729+0000 7f681555e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T21:08:23.945 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:23.944+0000 7f681555e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T21:08:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1212448855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T21:08:24.089 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:23.667+0000 7f9936bdc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T21:08:24.090 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:23.862+0000 7f9936bdc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:24.198 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:24.147+0000 7f681555e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T21:08:24.515 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T21:08:24.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T21:08:24.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T21:08:24.516 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T21:08:24.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:24.516 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T21:08:24.532 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T21:08:24.532 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T21:08:24.532 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T21:08:24.533 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41973sec preferred_lft 41973sec 2024-04-25T21:08:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-25T21:08:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T21:08:24.534 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T21:08:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T21:08:24.534 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T21:08:24.534 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T21:08:24.534 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T21:08:24.535 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T21:08:24.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:24.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:24.727 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T21:08:24.753 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T21:08:24.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:24.755 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T21:08:24.825 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T21:08:24.826 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T21:08:24.826 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T21:08:24.826 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T21:08:24.826 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T21:08:24.826 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T21:08:24.827 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T21:08:24.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T21:08:24.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T21:08:24.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T21:08:24.827 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T21:08:24.839 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:24.463+0000 7f9936bdc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T21:08:24.868 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:24 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:24.452+0000 7efd7536c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T21:08:24.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:25.032 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:24.793+0000 7f681555e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T21:08:25.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:25.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:25.116 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T21:08:25.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:25.133 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T21:08:25.162 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:24 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:24.868+0000 7efd7536c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T21:08:25.162 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:24 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:24.963+0000 7efd7536c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T21:08:25.162 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:25 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:25.162+0000 7efd7536c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T21:08:25.183 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:24.883+0000 7f9936bdc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T21:08:25.183 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:24.980+0000 7f9936bdc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T21:08:25.183 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:25.183+0000 7f9936bdc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T21:08:25.187 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T21:08:25.187 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T21:08:25.188 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T21:08:25.188 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T21:08:25.188 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T21:08:25.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:25.286 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:25 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:25.253+0000 7f681555e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T21:08:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:25.308 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T21:08:25.308 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T21:08:25.308 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T21:08:25.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T21:08:25.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T21:08:25.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T21:08:25.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T21:08:25.309 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T21:08:25.309 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T21:08:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:25.436 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T21:08:25.436 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T21:08:25.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T21:08:25.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T21:08:25.437 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T21:08:25.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:25.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:25.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T21:08:25.517 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:25.532 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T21:08:25.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T21:08:25.532 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:25.588 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T21:08:25.588 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T21:08:25.588 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/73dd5cfa-7e0d-4ed9-9126-8c1a699c1fe8 2024-04-25T21:08:25.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:25.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/73dd5cfa-7e0d-4ed9-9126-8c1a699c1fe8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T21:08:25.589 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:25 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:25.361+0000 7f681555e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T21:08:25.589 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:25 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:25.588+0000 7f681555e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T21:08:25.644 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T21:08:25.644 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T21:08:25.644 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:invalid new device string format 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.98:3300,172.21.15.177:3300" 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.37:3300,172.21.15.98:3300,172.21.15.177:3300:/volumes/qa/sv_1/73dd5cfa-7e0d-4ed9-9126-8c1a699c1fe8 2024-04-25T21:08:25.761 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=1788dafa-0346-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T21:08:25.762 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T21:08:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-25T21:08:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T21:08:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T21:08:25.762 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T21:08:25.764 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T21:08:25.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:25.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:25.794 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T21:08:25.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T21:08:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:25.894 INFO:teuthology.orchestra.run.smithi037.stdout:client24851 2024-04-25T21:08:25.895 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T21:08:25.895 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:08:25.911 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T21:08:26.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T21:08:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:26.010 INFO:teuthology.orchestra.run.smithi037.stdout:client24851 2024-04-25T21:08:26.011 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T21:08:26.018 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:25 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:25.682+0000 7efd7536c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T21:08:26.018 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:25 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:25.773+0000 7efd7536c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T21:08:26.044 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:25.708+0000 7f9936bdc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T21:08:26.044 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:25.802+0000 7f9936bdc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T21:08:26.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T21:08:26.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:26.067 INFO:teuthology.orchestra.run.smithi037.stdout:client24851 2024-04-25T21:08:26.068 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1788dafa-0346-11ef-bc93-c7b262605968.client24851/status') 2024-04-25T21:08:26.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T21:08:26.127 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24851 (3)172.21.15.37:0/501618326 2024-04-25T21:08:26.127 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-25T21:08:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-25T21:08:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-25T21:08:26.127 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 2.3292e-05 s, 3.0 MB/s 2024-04-25T21:08:26.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T21:08:26.135 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T21:08:26.144 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:08:26.144 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:08:26.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T21:08:26.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- 172.21.15.37:0/2650618023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 msgr2=0x7f0000101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 --2- 172.21.15.37:0/2650618023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f0000101280 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efff4009900 tx=0x7efff402a410 comp rx=0 tx=0).stop 2024-04-25T21:08:26.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- 172.21.15.37:0/2650618023 shutdown_connections 2024-04-25T21:08:26.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 --2- 172.21.15.37:0/2650618023 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 0x7f0000109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 --2- 172.21.15.37:0/2650618023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 --2- 172.21.15.37:0/2650618023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f0000101280 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- 172.21.15.37:0/2650618023 >> 172.21.15.37:0/2650618023 conn(0x7f00000fc420 msgr2=0x7f00000fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- 172.21.15.37:0/2650618023 shutdown_connections 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- 172.21.15.37:0/2650618023 wait complete. 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 Processor -- start 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.249+0000 7f0006f44640 1 -- start start 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 0x7f000019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f000010c9f0 con 0x7f0000100e60 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f000010c870 con 0x7f0000102a00 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f000010cb70 con 0x7f0000102040 2024-04-25T21:08:26.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60242/0 (socket says 172.21.15.37:60242) 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 -- 172.21.15.37:0/2186269416 learned_addr learned my addr 172.21.15.37:0/2186269416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7efffffff640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f00054e1640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 0x7f000019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 msgr2=0x7f00001986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 msgr2=0x7f000019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 0x7f000019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 -- 172.21.15.37:0/2186269416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000019fdd0 con 0x7f0000100e60 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7efffffff640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0004ce0640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff403f030 con 0x7f0000100e60 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4031910 con 0x7f0000100e60 2024-04-25T21:08:26.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00001a0060 con 0x7f0000100e60 2024-04-25T21:08:26.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.250+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4035070 con 0x7f0000100e60 2024-04-25T21:08:26.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.251+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001a05a0 con 0x7f0000100e60 2024-04-25T21:08:26.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.252+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00001a0800 con 0x7f0000100e60 2024-04-25T21:08:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.253+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7efff4042030 con 0x7f0000100e60 2024-04-25T21:08:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.253+0000 7efffdffb640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 0x7efff00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.253+0000 7efffffff640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 msgr2=0x7efff00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.253+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7efff40f0890 con 0x7f0000100e60 2024-04-25T21:08:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.253+0000 7efffffff640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 0x7efff00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T21:08:26.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.258+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efff40bcc30 con 0x7f0000100e60 2024-04-25T21:08:26.339 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:26.044+0000 7f9936bdc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T21:08:26.340 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:26.180+0000 7f9936bdc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T21:08:26.340 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:26.274+0000 7f9936bdc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T21:08:26.361 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:26.017+0000 7efd7536c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T21:08:26.361 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:26.151+0000 7efd7536c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T21:08:26.362 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:26.245+0000 7efd7536c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T21:08:26.362 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:26.361+0000 7efd7536c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.454+0000 7efffffff640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 msgr2=0x7efff00a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:26.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.454+0000 7efffffff640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 0x7efff00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T21:08:26.484 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.161+0000 7f681555e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T21:08:26.485 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.263+0000 7f681555e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T21:08:26.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.486+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f00001a09d0 con 0x7f0000100e60 2024-04-25T21:08:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.538+0000 7efffdffb640 1 -- 172.21.15.37:0/2186269416 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7efff40c0f30 con 0x7f0000100e60 2024-04-25T21:08:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 msgr2=0x7efff00a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:08:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 0x7efff00a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 msgr2=0x7f00001981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff40042f0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 shutdown_connections 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7efff00a3d20 0x7efff00a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0000102a00 0x7f000019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0000102040 0x7f00001986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 --2- 172.21.15.37:0/2186269416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0000100e60 0x7f00001981b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 >> 172.21.15.37:0/2186269416 conn(0x7f00000fc420 msgr2=0x7f00000fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 shutdown_connections 2024-04-25T21:08:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.541+0000 7f0006f44640 1 -- 172.21.15.37:0/2186269416 wait complete. 2024-04-25T21:08:26.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T21:08:26.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- 172.21.15.37:0/153853881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 msgr2=0x7f0cc4101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 --2- 172.21.15.37:0/153853881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4101500 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009900 tx=0x7f0cb402a410 comp rx=0 tx=0).stop 2024-04-25T21:08:26.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- 172.21.15.37:0/153853881 shutdown_connections 2024-04-25T21:08:26.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 --2- 172.21.15.37:0/153853881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cc4101a40 0x7f0cc4109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 --2- 172.21.15.37:0/153853881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4101500 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 --2- 172.21.15.37:0/153853881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cc4102450 0x7f0cc4102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- 172.21.15.37:0/153853881 >> 172.21.15.37:0/153853881 conn(0x7f0cc40fbbb0 msgr2=0x7f0cc40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:26.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- 172.21.15.37:0/153853881 shutdown_connections 2024-04-25T21:08:26.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- 172.21.15.37:0/153853881 wait complete. 2024-04-25T21:08:26.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 Processor -- start 2024-04-25T21:08:26.660 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T21:08:26.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.659+0000 7f0cc9308640 1 -- start start 2024-04-25T21:08:26.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4104e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cc4101a40 0x7f0cc4103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cc4102450 0x7f0cc41039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410c9b0 con 0x7f0cc41010a0 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410c830 con 0x7f0cc4102450 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc410cb30 con 0x7f0cc4101a40 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4104e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4104e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60272/0 (socket says 172.21.15.37:60272) 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 -- 172.21.15.37:0/1124236474 learned_addr learned my addr 172.21.15.37:0/1124236474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc37fe640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cc4102450 0x7f0cc41039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cc4101a40 0x7f0cc4103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cc4101a40 msgr2=0x7f0cc4103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cc4101a40 0x7f0cc4103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cc4102450 msgr2=0x7f0cc41039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0cc4102450 0x7f0cc41039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:26.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc4104260 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc2ffd640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4104e50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e9f0 tx=0x7f0cb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb800cd80 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb80054d0 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.660+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc41a2230 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.661+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8010690 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.661+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc41a2700 con 0x7f0cc41010a0 2024-04-25T21:08:26.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.662+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc404fce0 con 0x7f0cc41010a0 2024-04-25T21:08:26.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.663+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0cb8004e90 con 0x7f0cc41010a0 2024-04-25T21:08:26.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.663+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:26.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.663+0000 7f0cc27fc640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 msgr2=0x7f0cac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:26.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.663+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8211+0+0 (secure 0 0 0) 0x7f0cb80d08a0 con 0x7f0cc41010a0 2024-04-25T21:08:26.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.663+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T21:08:26.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.668+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cb809d290 con 0x7f0cc41010a0 2024-04-25T21:08:26.697 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T21:08:26.697 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:26.391+0000 7f9936bdc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:26.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T21:08:26.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.524+0000 7f681555e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T21:08:26.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.672+0000 7f681555e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T21:08:26.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.864+0000 7f0cc27fc640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 msgr2=0x7f0cac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:26.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.864+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T21:08:26.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:26.883+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0cc40fdd30 con 0x7f0cac0a3ca0 2024-04-25T21:08:26.917 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:26.660+0000 7efd7536c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T21:08:27.215 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.775+0000 7f681555e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T21:08:27.215 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:26.900+0000 7f681555e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T21:08:27.215 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:27.214+0000 7f681555e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T21:08:27.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:27.265+0000 7f0cc27fc640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 msgr2=0x7f0cac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:27.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:27.265+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T21:08:27.544 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:27.258+0000 7efd7536c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T21:08:27.545 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y[33962]: 2024-04-25T21:08:27.474+0000 7efd7536c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T21:08:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:27.557+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f0cb8099200 con 0x7f0cc41010a0 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: Standby manager daemon y restarted 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: Standby manager daemon y started 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T21:08:27.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: Standby manager daemon z restarted 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: Standby manager daemon z started 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-mon[33043]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:26.697+0000 7f9936bdc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:27.298+0000 7f9936bdc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T21:08:27.591 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:08:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z[34893]: 2024-04-25T21:08:27.513+0000 7f9936bdc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: Standby manager daemon y restarted 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: Standby manager daemon y started 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T21:08:27.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: Standby manager daemon z restarted 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: Standby manager daemon z started 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T21:08:27.861 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-mon[29195]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2186269416' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T21:08:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: Standby manager daemon y restarted 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: Standby manager daemon y started 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.98:0/1246921630' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: Standby manager daemon z restarted 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: Standby manager daemon z started 2024-04-25T21:08:27.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T21:08:27.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T21:08:27.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T21:08:27.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:27 smithi098 ceph-mon[32569]: from='mgr.? 172.21.15.177:0/3921330310' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T21:08:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:28.066+0000 7f0cc27fc640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 msgr2=0x7f0cac0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2407888079 2024-04-25T21:08:28.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:28.066+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T21:08:28.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:28.103+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f0cb80a3730 con 0x7f0cc41010a0 2024-04-25T21:08:28.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:28.103+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 msgr2=0x7f0cac0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T21:08:28.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:28.103+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/2407888079,v1:172.21.15.37:6801/2407888079] conn(0x7f0cac0a3ca0 0x7f0cac0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.234 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:27.859+0000 7f681555e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T21:08:28.235 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.090+0000 7f681555e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.351+0000 7f6777e98700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.351+0000 7f6777e98700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.351+0000 7f6777e98700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.351+0000 7f6777e98700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.351+0000 7f6777e98700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.355+0000 7f6774e52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.355+0000 7f6774e52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.355+0000 7f6774e52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.355+0000 7f6774e52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.553 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:28.355+0000 7f6774e52700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T21:08:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: Active manager daemon x restarted 2024-04-25T21:08:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: Activating manager daemon x 2024-04-25T21:08:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T21:08:28.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: mgrmap e21: x(active, starting, since 0.0112539s), standbys: y, z 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:28.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:08:28.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: Manager daemon x is now available 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T21:08:28.843 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:28 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T21:08:28.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T21:08:28.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: Active manager daemon x restarted 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: Activating manager daemon x 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: mgrmap e21: x(active, starting, since 0.0112539s), standbys: y, z 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:08:28.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:08:28.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: Manager daemon x is now available 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T21:08:28.920 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:28 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T21:08:28.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T21:08:28.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: Active manager daemon x restarted 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: Activating manager daemon x 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: mgrmap e21: x(active, starting, since 0.0112539s), standbys: y, z 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T21:08:28.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T21:08:28.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T21:08:28.988 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T21:08:28.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: Manager daemon x is now available 2024-04-25T21:08:28.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:28.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T21:08:28.989 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:28 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T21:08:29.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.109+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mon.0 v2:172.21.15.37:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0cb804e360 con 0x7f0cc41010a0 2024-04-25T21:08:29.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.110+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0cac035660 0x7f0cac035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.110+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0cb8012ec0 con 0x7f0cac035660 2024-04-25T21:08:29.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.114+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0cac035660 0x7f0cac035a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.117+0000 7f0cc27fc640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0cac035660 0x7f0cac035a40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb40314a0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.136+0000 7f0cabfff640 1 -- 172.21.15.37:0/1124236474 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f0cb8012ec0 con 0x7f0cac035660 2024-04-25T21:08:29.136 INFO:teuthology.orchestra.run.smithi037.stdout:{} 2024-04-25T21:08:29.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0cac035660 msgr2=0x7f0cac035a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0cac035660 0x7f0cac035a40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000c00 tx=0x7f0cb40314a0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 msgr2=0x7f0cc4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 --2- 172.21.15.37:0/1124236474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cc41010a0 0x7f0cc4104e50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0cb800e9f0 tx=0x7f0cb800edf0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc37fe640 1 -- 172.21.15.37:0/1124236474 reap_dead start 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 shutdown_connections 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 >> 172.21.15.37:0/1124236474 conn(0x7f0cc40fbbb0 msgr2=0x7f0cc40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 shutdown_connections 2024-04-25T21:08:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.138+0000 7f0cc9308640 1 -- 172.21.15.37:0/1124236474 wait complete. 2024-04-25T21:08:29.151 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T21:08:29.162 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T21:08:29.172 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:08:29.172 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:08:29.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T21:08:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 -- 172.21.15.37:0/4243290996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328105400 msgr2=0x7ff3281077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 --2- 172.21.15.37:0/4243290996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328105400 0x7ff3281077f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff310009900 tx=0x7ff31002a410 comp rx=0 tx=0).stop 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 -- 172.21.15.37:0/4243290996 shutdown_connections 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 --2- 172.21.15.37:0/4243290996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328107d30 0x7ff32810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 --2- 172.21.15.37:0/4243290996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328105400 0x7ff3281077f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 --2- 172.21.15.37:0/4243290996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328102ad0 0x7ff328104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.282+0000 7ff32e689640 1 -- 172.21.15.37:0/4243290996 >> 172.21.15.37:0/4243290996 conn(0x7ff3280fc420 msgr2=0x7ff3280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- 172.21.15.37:0/4243290996 shutdown_connections 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- 172.21.15.37:0/4243290996 wait complete. 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 Processor -- start 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- start start 2024-04-25T21:08:29.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 0x7ff328198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328105400 0x7ff3281986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810db60 con 0x7ff328107d30 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810d9e0 con 0x7ff328105400 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.283+0000 7ff32e689640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810dce0 con 0x7ff328102ad0 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50018/0 (socket says 172.21.15.37:50018) 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff327fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 0x7ff328198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 -- 172.21.15.37:0/381054147 learned_addr learned my addr 172.21.15.37:0/381054147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff3277fe640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328105400 0x7ff3281986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 -- 172.21.15.37:0/381054147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 msgr2=0x7ff328198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 0x7ff328198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 -- 172.21.15.37:0/381054147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328105400 msgr2=0x7ff3281986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328105400 0x7ff3281986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 -- 172.21.15.37:0/381054147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32819fd80 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff327fff640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 0x7ff328198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32cc26640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eb10 tx=0x7ff32000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320004b20 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3281a0070 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff320004cc0 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320010670 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.284+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3281a0540 con 0x7ff328107d30 2024-04-25T21:08:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.285+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3280fdf10 con 0x7ff328107d30 2024-04-25T21:08:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.287+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ff320005000 con 0x7ff328107d30 2024-04-25T21:08:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.287+0000 7ff3257fa640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 0x7ff3140a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.287+0000 7ff327fff640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 0x7ff3140a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.288+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff3200d0880 con 0x7ff328107d30 2024-04-25T21:08:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.288+0000 7ff327fff640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 0x7ff3140a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff318009fc0 tx=0x7ff318005f90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:29.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.295+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff320053ff0 con 0x7ff328107d30 2024-04-25T21:08:29.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:29.125+0000 7f6771e4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:29.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:29.125+0000 7f6771e4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:29.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:29.125+0000 7f6771e4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:29.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:29.125+0000 7f6771e4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:29.486 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:08:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:08:29.125+0000 7f6771e4c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:08:29.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.506+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff328062390 con 0x7ff3140a3a60 2024-04-25T21:08:29.570 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-04-25T21:08:29.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.569+0000 7ff3257fa640 1 -- 172.21.15.37:0/381054147 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff328062390 con 0x7ff3140a3a60 2024-04-25T21:08:29.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 msgr2=0x7ff3140a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 0x7ff3140a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff318009fc0 tx=0x7ff318005f90 comp rx=0 tx=0).stop 2024-04-25T21:08:29.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 msgr2=0x7ff32819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff32000eb10 tx=0x7ff32000c2d0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 shutdown_connections 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff328107d30 0x7ff32819f710 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff328105400 0x7ff3281986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff3140a3a60 0x7ff3140a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 --2- 172.21.15.37:0/381054147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff328102ad0 0x7ff328198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 >> 172.21.15.37:0/381054147 conn(0x7ff3280fc420 msgr2=0x7ff328103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 shutdown_connections 2024-04-25T21:08:29.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.572+0000 7ff32e689640 1 -- 172.21.15.37:0/381054147 wait complete. 2024-04-25T21:08:29.583 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 -- 172.21.15.37:0/3189035545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4170075f90 msgr2=0x7f41700743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 --2- 172.21.15.37:0/3189035545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4170075f90 0x7f41700743e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4158009930 tx=0x7f415802a420 comp rx=0 tx=0).stop 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 -- 172.21.15.37:0/3189035545 shutdown_connections 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 --2- 172.21.15.37:0/3189035545 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f417010e5d0 0x7f41701109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 --2- 172.21.15.37:0/3189035545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f4170074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 --2- 172.21.15.37:0/3189035545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4170075f90 0x7f41700743e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 -- 172.21.15.37:0/3189035545 >> 172.21.15.37:0/3189035545 conn(0x7f41700fd570 msgr2=0x7f41700ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:29.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 -- 172.21.15.37:0/3189035545 shutdown_connections 2024-04-25T21:08:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.694+0000 7f4175005640 1 -- 172.21.15.37:0/3189035545 wait complete. 2024-04-25T21:08:29.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 Processor -- start 2024-04-25T21:08:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 -- start start 2024-04-25T21:08:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4170075f90 0x7f417019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 0x7f41701a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4170114100 con 0x7f417010e5d0 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4170113f80 con 0x7f41700749b0 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.695+0000 7f4175005640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4170114280 con 0x7f4170075f90 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 0x7f41701a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:51580/0 (socket says 172.21.15.37:51580) 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416e59c640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4170075f90 0x7f417019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416ed9d640 1 -- 172.21.15.37:0/4112614083 learned_addr learned my addr 172.21.15.37:0/4112614083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 -- 172.21.15.37:0/4112614083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4170075f90 msgr2=0x7f417019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4170075f90 0x7f417019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 -- 172.21.15.37:0/4112614083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 msgr2=0x7f417019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 -- 172.21.15.37:0/4112614083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41701a63b0 con 0x7f417010e5d0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416ed9d640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.696+0000 7f416f59e640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 0x7f41701a5d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f416800cab0 tx=0x7f416800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.697+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416800ae20 con 0x7f417010e5d0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.697+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416800f900 con 0x7f417010e5d0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.697+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41701a66a0 con 0x7f417010e5d0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.697+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680189f0 con 0x7f417010e5d0 2024-04-25T21:08:29.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.697+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41701a6bf0 con 0x7f417010e5d0 2024-04-25T21:08:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.698+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41701047a0 con 0x7f417010e5d0 2024-04-25T21:08:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.699+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f416800f450 con 0x7f417010e5d0 2024-04-25T21:08:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.699+0000 7f4167fff640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 0x7f415c0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.700+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f41680d07c0 con 0x7f417010e5d0 2024-04-25T21:08:29.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.700+0000 7f416ed9d640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 0x7f415c0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.700+0000 7f416ed9d640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 0x7f415c0a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4158004770 tx=0x7f41580314c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.704+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4168014030 con 0x7f417010e5d0 2024-04-25T21:08:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.914+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4170062430 con 0x7f415c0a3ae0 2024-04-25T21:08:29.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.976+0000 7f4167fff640 1 -- 172.21.15.37:0/4112614083 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4170062430 con 0x7f415c0a3ae0 2024-04-25T21:08:29.977 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-04-25T21:08:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 msgr2=0x7f415c0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 0x7f415c0a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4158004770 tx=0x7f41580314c0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 msgr2=0x7f41701a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 0x7f41701a5d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f416800cab0 tx=0x7f416800cf70 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 shutdown_connections 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f417010e5d0 0x7f41701a5d40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f415c0a3ae0 0x7f415c0a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4170075f90 0x7f417019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 --2- 172.21.15.37:0/4112614083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41700749b0 0x7f417019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 >> 172.21.15.37:0/4112614083 conn(0x7f41700fd570 msgr2=0x7f4170101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 shutdown_connections 2024-04-25T21:08:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:29.978+0000 7f4175005640 1 -- 172.21.15.37:0/4112614083 wait complete. 2024-04-25T21:08:29.990 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T21:08:30.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 -- 172.21.15.37:0/1875396700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 msgr2=0x7f371c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 --2- 172.21.15.37:0/1875396700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c1023f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f370c009900 tx=0x7f370c02a410 comp rx=0 tx=0).stop 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 -- 172.21.15.37:0/1875396700 shutdown_connections 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 --2- 172.21.15.37:0/1875396700 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 0x7f371c10a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 --2- 172.21.15.37:0/1875396700 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 0x7f371c103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 --2- 172.21.15.37:0/1875396700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c1023f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 -- 172.21.15.37:0/1875396700 >> 172.21.15.37:0/1875396700 conn(0x7f371c0fd540 msgr2=0x7f371c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 -- 172.21.15.37:0/1875396700 shutdown_connections 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.108+0000 7f3721a6b640 1 -- 172.21.15.37:0/1875396700 wait complete. 2024-04-25T21:08:30.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 Processor -- start 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 -- start start 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 0x7f371c19a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 0x7f371c1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10db60 con 0x7f371c101fd0 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10d9e0 con 0x7f371c103b80 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f3721a6b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f371c10dce0 con 0x7f371c1031c0 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50052/0 (socket says 172.21.15.37:50052) 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371b7fe640 1 -- 172.21.15.37:0/3034227407 learned_addr learned my addr 172.21.15.37:0/3034227407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371affd640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 0x7f371c19a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371bfff640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 0x7f371c1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371b7fe640 1 -- 172.21.15.37:0/3034227407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 msgr2=0x7f371c19a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.109+0000 7f371b7fe640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 0x7f371c19a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f371b7fe640 1 -- 172.21.15.37:0/3034227407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 msgr2=0x7f371c1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f371b7fe640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 0x7f371c1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f371b7fe640 1 -- 172.21.15.37:0/3034227407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371c1a1fc0 con 0x7f371c101fd0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f371b7fe640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f370c031500 tx=0x7f370c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c03f070 con 0x7f371c101fd0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371c1a2250 con 0x7f371c101fd0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370c031c50 con 0x7f371c101fd0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c035030 con 0x7f371c101fd0 2024-04-25T21:08:30.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.110+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371c1a2790 con 0x7f371c101fd0 2024-04-25T21:08:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.111+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371c1a2ab0 con 0x7f371c101fd0 2024-04-25T21:08:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.112+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f370c042070 con 0x7f371c101fd0 2024-04-25T21:08:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.113+0000 7f3718ff9640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 0x7f37040a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.113+0000 7f371affd640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 0x7f37040a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.113+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f370c0f07b0 con 0x7f371c101fd0 2024-04-25T21:08:30.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.114+0000 7f371affd640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 0x7f37040a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f371c19b8b0 tx=0x7f3710005f90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:30.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.118+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f370c0bcd40 con 0x7f371c101fd0 2024-04-25T21:08:30.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: [25/Apr/2024:21:08:29] ENGINE Bus STARTING 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: [25/Apr/2024:21:08:29] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: [25/Apr/2024:21:08:29] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: [25/Apr/2024:21:08:29] ENGINE Bus STARTED 2024-04-25T21:08:30.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:30 smithi177 ceph-mon[33043]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:30.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.363+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f371c0758a0 con 0x7f37040a3a60 2024-04-25T21:08:30.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.372+0000 7f3718ff9640 1 -- 172.21.15.37:0/3034227407 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f371c0758a0 con 0x7f37040a3a60 2024-04-25T21:08:30.372 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T21:08:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.375+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 msgr2=0x7f37040a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 0x7f37040a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f371c19b8b0 tx=0x7f3710005f90 comp rx=0 tx=0).stop 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 msgr2=0x7f371c19a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f370c031500 tx=0x7f370c003c10 comp rx=0 tx=0).stop 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 shutdown_connections 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f371c103b80 0x7f371c1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f37040a3a60 0x7f37040a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f371c1031c0 0x7f371c19a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 --2- 172.21.15.37:0/3034227407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f371c101fd0 0x7f371c19a3f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 >> 172.21.15.37:0/3034227407 conn(0x7f371c0fd540 msgr2=0x7f371c074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:30.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 shutdown_connections 2024-04-25T21:08:30.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.376+0000 7f3721a6b640 1 -- 172.21.15.37:0/3034227407 wait complete. 2024-04-25T21:08:30.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: [25/Apr/2024:21:08:29] ENGINE Bus STARTING 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: [25/Apr/2024:21:08:29] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: [25/Apr/2024:21:08:29] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: [25/Apr/2024:21:08:29] ENGINE Bus STARTED 2024-04-25T21:08:30.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:30 smithi037 ceph-mon[29195]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:30.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: mgrmap e22: x(active, since 1.02019s), standbys: y, z 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: [25/Apr/2024:21:08:29] ENGINE Bus STARTING 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: [25/Apr/2024:21:08:29] ENGINE Serving on https://172.21.15.37:7150 2024-04-25T21:08:30.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: [25/Apr/2024:21:08:29] ENGINE Serving on http://172.21.15.37:8765 2024-04-25T21:08:30.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: [25/Apr/2024:21:08:29] ENGINE Bus STARTED 2024-04-25T21:08:30.418 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:30 smithi098 ceph-mon[32569]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 -- 172.21.15.37:0/316182356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56001011e0 msgr2=0x7f5600101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 --2- 172.21.15.37:0/316182356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56001011e0 0x7f5600101640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009900 tx=0x7f55f002a410 comp rx=0 tx=0).stop 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 -- 172.21.15.37:0/316182356 shutdown_connections 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 --2- 172.21.15.37:0/316182356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5600101b80 0x7f5600102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 --2- 172.21.15.37:0/316182356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56001011e0 0x7f5600101640 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 --2- 172.21.15.37:0/316182356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600100880 0x7f5600100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.499+0000 7f5607e43640 1 -- 172.21.15.37:0/316182356 >> 172.21.15.37:0/316182356 conn(0x7f56000fc420 msgr2=0x7f56000fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- 172.21.15.37:0/316182356 shutdown_connections 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- 172.21.15.37:0/316182356 wait complete. 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 Processor -- start 2024-04-25T21:08:30.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- start start 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56001011e0 0x7f560019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600101b80 0x7f56001a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f560010fd00 con 0x7f5600100880 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f560010fb80 con 0x7f56001011e0 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5607e43640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f560010fe80 con 0x7f5600101b80 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5605bdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5605bdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50090/0 (socket says 172.21.15.37:50090) 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f5605bdf640 1 -- 172.21.15.37:0/3004952805 learned_addr learned my addr 172.21.15.37:0/3004952805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f56063e0640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600101b80 0x7f56001a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.500+0000 7f56053de640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56001011e0 0x7f560019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 -- 172.21.15.37:0/3004952805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600101b80 msgr2=0x7f56001a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600101b80 0x7f56001a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 -- 172.21.15.37:0/3004952805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56001011e0 msgr2=0x7f560019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56001011e0 0x7f560019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 -- 172.21.15.37:0/3004952805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a1f80 con 0x7f5600100880 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5605bdf640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00e9f0 tx=0x7f55fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00cd80 con 0x7f5600100880 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0054d0 con 0x7f5600100880 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a2270 con 0x7f5600100880 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc010690 con 0x7f5600100880 2024-04-25T21:08:30.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.501+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a2790 con 0x7f5600100880 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.502+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560010b5d0 con 0x7f5600100880 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.503+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f55fc004e90 con 0x7f5600100880 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.504+0000 7f55f6ffd640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 0x7f55ec0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.504+0000 7f56053de640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 0x7f55ec0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.504+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f55fc0d0480 con 0x7f5600100880 2024-04-25T21:08:30.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.505+0000 7f56053de640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 0x7f55ec0a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f00314a0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:30.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.509+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55fc09d060 con 0x7f5600100880 2024-04-25T21:08:30.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.719+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5600062390 con 0x7f55ec0a3ae0 2024-04-25T21:08:30.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.728+0000 7f55f6ffd640 1 -- 172.21.15.37:0/3004952805 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5600062390 con 0x7f55ec0a3ae0 2024-04-25T21:08:30.728 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 msgr2=0x7f55ec0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 0x7f55ec0a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55f0000c00 tx=0x7f55f00314a0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 msgr2=0x7f560019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00e9f0 tx=0x7f55fc00edf0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 shutdown_connections 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55ec0a3ae0 0x7f55ec0a5fa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5600101b80 0x7f56001a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56001011e0 0x7f560019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 --2- 172.21.15.37:0/3004952805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5600100880 0x7f560019a360 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 >> 172.21.15.37:0/3004952805 conn(0x7f56000fc420 msgr2=0x7f56000ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 shutdown_connections 2024-04-25T21:08:30.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:30.730+0000 7f5607e43640 1 -- 172.21.15.37:0/3004952805 wait complete. 2024-04-25T21:08:30.741 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T21:08:30.767 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T21:08:30.778 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T21:08:30.821 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:08:30.821 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T21:08:30.823 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T21:08:30.861 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T21:08:30.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:08:31.136 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:31 smithi177 ceph-mon[33043]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:31 smithi037 ceph-mon[29195]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:31.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:31 smithi098 ceph-mon[32569]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.583 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.584 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:32 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.668 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:32 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:32.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:32 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 -- 172.21.15.37:0/3722522286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 msgr2=0x7f4e8c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 --2- 172.21.15.37:0/3722522286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c108e40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009990 tx=0x7f4e80009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 -- 172.21.15.37:0/3722522286 shutdown_connections 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 --2- 172.21.15.37:0/3722522286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 0x7f4e8c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 --2- 172.21.15.37:0/3722522286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c108e40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 --2- 172.21.15.37:0/3722522286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 -- 172.21.15.37:0/3722522286 >> 172.21.15.37:0/3722522286 conn(0x7f4e8c078980 msgr2=0x7f4e8c078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:33.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 -- 172.21.15.37:0/3722522286 shutdown_connections 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.994+0000 7f4e92e53700 1 -- 172.21.15.37:0/3722522286 wait complete. 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 Processor -- start 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 -- start start 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 0x7f4e8c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c112be0 con 0x7f4e8c108a60 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c112a60 con 0x7f4e8c06bae0 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e92e53700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c112d60 con 0x7f4e8c06b140 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:54462/0 (socket says 172.21.15.37:54462) 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 -- 172.21.15.37:0/2724727980 learned_addr learned my addr 172.21.15.37:0/2724727980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:33.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e913f0700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e8bfff700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 0x7f4e8c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 -- 172.21.15.37:0/2724727980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 msgr2=0x7f4e8c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 0x7f4e8c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 -- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 msgr2=0x7f4e8c19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.995+0000 7f4e90bef700 1 -- 172.21.15.37:0/2724727980 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19e120 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e90bef700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e8000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80027070 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c19e3b0 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80022470 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80017650 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.996+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1a5c90 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.997+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c1041a0 con 0x7f4e8c06b140 2024-04-25T21:08:33.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.998+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4e8002a070 con 0x7f4e8c06b140 2024-04-25T21:08:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.999+0000 7f4e89ffb700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 0x7f4e7c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.999+0000 7f4e8bfff700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 0x7f4e7c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:32.999+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4e800d16d0 con 0x7f4e8c06b140 2024-04-25T21:08:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.000+0000 7f4e8bfff700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 0x7f4e7c0a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c079130 tx=0x7f4e78009260 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.003+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e80013070 con 0x7f4e8c06b140 2024-04-25T21:08:33.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.234+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e8c19a470 con 0x7f4e7c0a3ed0 2024-04-25T21:08:33.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.243+0000 7f4e89ffb700 1 -- 172.21.15.37:0/2724727980 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258045 (secure 0 0 0) 0x7f4e8c19a470 con 0x7f4e7c0a3ed0 2024-04-25T21:08:33.247 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:33.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 msgr2=0x7f4e7c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 0x7f4e7c0a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c079130 tx=0x7f4e78009260 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 msgr2=0x7f4e8c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e80000c00 tx=0x7f4e8000bfd0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 shutdown_connections 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4e7c0a3ed0 0x7f4e7c0a6390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e8c108a60 0x7f4e8c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4e8c06b140 0x7f4e8c199110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 --2- 172.21.15.37:0/2724727980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e8c06bae0 0x7f4e8c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 >> 172.21.15.37:0/2724727980 conn(0x7f4e8c078980 msgr2=0x7f4e8c0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 shutdown_connections 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:33.248+0000 7f4e92e53700 1 -- 172.21.15.37:0/2724727980 wait complete. 2024-04-25T21:08:33.250 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:08:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:08:33.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:33.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:33.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T21:08:33.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T21:08:33.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T21:08:33.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:33 smithi177 ceph-mon[33043]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T21:08:33.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:33 smithi098 ceph-mon[32569]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:08:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Updating smithi098:/etc/ceph/ceph.conf 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: Updating smithi177:/etc/ceph/ceph.conf 2024-04-25T21:08:33.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:33 smithi037 ceph-mon[29195]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T21:08:34.028 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T21:08:32.113365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191641,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2167,"num_read_kb":1251,"num_write":565,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":947,"ondisk_log_size":947,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365736,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576088,"statfs":{"total":1151940427776,"available":1151565914112,"internally_reserved":0,"allocated":12681216,"data_stored":8897937,"data_compressed":189552,"data_compressed_allocated":245760,"data_compressed_original":2891776,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987836"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362557+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:08:28.362557+0000","last_peered":"2024-04-25T21:08:28.362557+0000","last_clean":"2024-04-25T21:08:28.362557+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:08:28.362557+0000","last_undegraded":"2024-04-25T21:08:28.362557+0000","last_fullsized":"2024-04-25T21:08:28.362557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362924+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:08:28.362924+0000","last_peered":"2024-04-25T21:08:28.362924+0000","last_clean":"2024-04-25T21:08:28.362924+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:08:28.362924+0000","last_undegraded":"2024-04-25T21:08:28.362924+0000","last_fullsized":"2024-04-25T21:08:28.362924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363987+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:08:28.363987+0000","last_peered":"2024-04-25T21:08:28.363987+0000","last_clean":"2024-04-25T21:08:28.363987+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:08:28.363987+0000","last_undegraded":"2024-04-25T21:08:28.363987+0000","last_fullsized":"2024-04-25T21:08:28.363987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195181+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:08:28.195181+0000","last_peered":"2024-04-25T21:08:28.195181+0000","last_clean":"2024-04-25T21:08:28.195181+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:08:28.195181+0000","last_undegraded":"2024-04-25T21:08:28.195181+0000","last_fullsized":"2024-04-25T21:08:28.195181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205898+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:08:28.205898+0000","last_peered":"2024-04-25T21:08:28.205898+0000","last_clean":"2024-04-25T21:08:28.205898+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:08:28.205898+0000","last_undegraded":"2024-04-25T21:08:28.205898+0000","last_fullsized":"2024-04-25T21:08:28.205898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357638+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:08:28.357638+0000","last_peered":"2024-04-25T21:08:28.357638+0000","last_clean":"2024-04-25T21:08:28.357638+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:08:28.357638+0000","last_undegraded":"2024-04-25T21:08:28.357638+0000","last_fullsized":"2024-04-25T21:08:28.357638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194847+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:08:28.194847+0000","last_peered":"2024-04-25T21:08:28.194847+0000","last_clean":"2024-04-25T21:08:28.194847+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:08:28.194847+0000","last_undegraded":"2024-04-25T21:08:28.194847+0000","last_fullsized":"2024-04-25T21:08:28.194847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101029+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:08:28.101029+0000","last_peered":"2024-04-25T21:08:28.101029+0000","last_clean":"2024-04-25T21:08:28.101029+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:08:28.101029+0000","last_undegraded":"2024-04-25T21:08:28.101029+0000","last_fullsized":"2024-04-25T21:08:28.101029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206257+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:08:28.206257+0000","last_peered":"2024-04-25T21:08:28.206257+0000","last_clean":"2024-04-25T21:08:28.206257+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:08:28.206257+0000","last_undegraded":"2024-04-25T21:08:28.206257+0000","last_fullsized":"2024-04-25T21:08:28.206257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358006+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:08:28.358006+0000","last_peered":"2024-04-25T21:08:28.358006+0000","last_clean":"2024-04-25T21:08:28.358006+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:08:28.358006+0000","last_undegraded":"2024-04-25T21:08:28.358006+0000","last_fullsized":"2024-04-25T21:08:28.358006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363955+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:08:28.363955+0000","last_peered":"2024-04-25T21:08:28.363955+0000","last_clean":"2024-04-25T21:08:28.363955+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:08:28.363955+0000","last_undegraded":"2024-04-25T21:08:28.363955+0000","last_fullsized":"2024-04-25T21:08:28.363955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360589+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:08:28.360589+0000","last_peered":"2024-04-25T21:08:28.360589+0000","last_clean":"2024-04-25T21:08:28.360589+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:08:28.360589+0000","last_undegraded":"2024-04-25T21:08:28.360589+0000","last_fullsized":"2024-04-25T21:08:28.360589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362450+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:08:28.362450+0000","last_peered":"2024-04-25T21:08:28.362450+0000","last_clean":"2024-04-25T21:08:28.362450+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:08:28.362450+0000","last_undegraded":"2024-04-25T21:08:28.362450+0000","last_fullsized":"2024-04-25T21:08:28.362450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362235+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:08:28.362235+0000","last_peered":"2024-04-25T21:08:28.362235+0000","last_clean":"2024-04-25T21:08:28.362235+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:08:28.362235+0000","last_undegraded":"2024-04-25T21:08:28.362235+0000","last_fullsized":"2024-04-25T21:08:28.362235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100450+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:08:28.100450+0000","last_peered":"2024-04-25T21:08:28.100450+0000","last_clean":"2024-04-25T21:08:28.100450+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:08:28.100450+0000","last_undegraded":"2024-04-25T21:08:28.100450+0000","last_fullsized":"2024-04-25T21:08:28.100450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362697+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:08:28.362697+0000","last_peered":"2024-04-25T21:08:28.362697+0000","last_clean":"2024-04-25T21:08:28.362697+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:08:28.362697+0000","last_undegraded":"2024-04-25T21:08:28.362697+0000","last_fullsized":"2024-04-25T21:08:28.362697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.099215+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:08:28.099215+0000","last_peered":"2024-04-25T21:08:28.099215+0000","last_clean":"2024-04-25T21:08:28.099215+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:08:28.099215+0000","last_undegraded":"2024-04-25T21:08:28.099215+0000","last_fullsized":"2024-04-25T21:08:28.099215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101330+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:08:28.101330+0000","last_peered":"2024-04-25T21:08:28.101330+0000","last_clean":"2024-04-25T21:08:28.101330+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:08:28.101330+0000","last_undegraded":"2024-04-25T21:08:28.101330+0000","last_fullsized":"2024-04-25T21:08:28.101330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":233,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358001+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:08:28.358001+0000","last_peered":"2024-04-25T21:08:28.358001+0000","last_clean":"2024-04-25T21:08:28.358001+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:08:28.358001+0000","last_undegraded":"2024-04-25T21:08:28.358001+0000","last_fullsized":"2024-04-25T21:08:28.358001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195064+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:08:28.195064+0000","last_peered":"2024-04-25T21:08:28.195064+0000","last_clean":"2024-04-25T21:08:28.195064+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:08:28.195064+0000","last_undegraded":"2024-04-25T21:08:28.195064+0000","last_fullsized":"2024-04-25T21:08:28.195064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195494+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:08:28.195494+0000","last_peered":"2024-04-25T21:08:28.195494+0000","last_clean":"2024-04-25T21:08:28.195494+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:08:28.195494+0000","last_undegraded":"2024-04-25T21:08:28.195494+0000","last_fullsized":"2024-04-25T21:08:28.195494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363253+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:08:28.363253+0000","last_peered":"2024-04-25T21:08:28.363253+0000","last_clean":"2024-04-25T21:08:28.363253+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:08:28.363253+0000","last_undegraded":"2024-04-25T21:08:28.363253+0000","last_fullsized":"2024-04-25T21:08:28.363253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195601+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:08:28.195601+0000","last_peered":"2024-04-25T21:08:28.195601+0000","last_clean":"2024-04-25T21:08:28.195601+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:08:28.195601+0000","last_undegraded":"2024-04-25T21:08:28.195601+0000","last_fullsized":"2024-04-25T21:08:28.195601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100566+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:08:28.100566+0000","last_peered":"2024-04-25T21:08:28.100566+0000","last_clean":"2024-04-25T21:08:28.100566+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:08:28.100566+0000","last_undegraded":"2024-04-25T21:08:28.100566+0000","last_fullsized":"2024-04-25T21:08:28.100566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362183+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:08:28.362183+0000","last_peered":"2024-04-25T21:08:28.362183+0000","last_clean":"2024-04-25T21:08:28.362183+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:08:28.362183+0000","last_undegraded":"2024-04-25T21:08:28.362183+0000","last_fullsized":"2024-04-25T21:08:28.362183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203110+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:08:28.203110+0000","last_peered":"2024-04-25T21:08:28.203110+0000","last_clean":"2024-04-25T21:08:28.203110+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:08:28.203110+0000","last_undegraded":"2024-04-25T21:08:28.203110+0000","last_fullsized":"2024-04-25T21:08:28.203110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":231,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.520530+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:08:28.520530+0000","last_peered":"2024-04-25T21:08:28.520530+0000","last_clean":"2024-04-25T21:08:28.520530+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:08:28.520530+0000","last_undegraded":"2024-04-25T21:08:28.520530+0000","last_fullsized":"2024-04-25T21:08:28.520530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":194,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195157+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:08:28.195157+0000","last_peered":"2024-04-25T21:08:28.195157+0000","last_clean":"2024-04-25T21:08:28.195157+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:08:28.195157+0000","last_undegraded":"2024-04-25T21:08:28.195157+0000","last_fullsized":"2024-04-25T21:08:28.195157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205818+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:08:28.205818+0000","last_peered":"2024-04-25T21:08:28.205818+0000","last_clean":"2024-04-25T21:08:28.205818+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:08:28.205818+0000","last_undegraded":"2024-04-25T21:08:28.205818+0000","last_fullsized":"2024-04-25T21:08:28.205818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203072+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:08:28.203072+0000","last_peered":"2024-04-25T21:08:28.203072+0000","last_clean":"2024-04-25T21:08:28.203072+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:08:28.203072+0000","last_undegraded":"2024-04-25T21:08:28.203072+0000","last_fullsized":"2024-04-25T21:08:28.203072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359792+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:08:28.359792+0000","last_peered":"2024-04-25T21:08:28.359792+0000","last_clean":"2024-04-25T21:08:28.359792+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:08:28.359792+0000","last_undegraded":"2024-04-25T21:08:28.359792+0000","last_fullsized":"2024-04-25T21:08:28.359792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194771+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:08:28.194771+0000","last_peered":"2024-04-25T21:08:28.194771+0000","last_clean":"2024-04-25T21:08:28.194771+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:08:28.194771+0000","last_undegraded":"2024-04-25T21:08:28.194771+0000","last_fullsized":"2024-04-25T21:08:28.194771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202732+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:08:28.202732+0000","last_peered":"2024-04-25T21:08:28.202732+0000","last_clean":"2024-04-25T21:08:28.202732+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:08:28.202732+0000","last_undegraded":"2024-04-25T21:08:28.202732+0000","last_fullsized":"2024-04-25T21:08:28.202732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098804+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:08:28.098804+0000","last_peered":"2024-04-25T21:08:28.098804+0000","last_clean":"2024-04-25T21:08:28.098804+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:08:28.098804+0000","last_undegraded":"2024-04-25T21:08:28.098804+0000","last_fullsized":"2024-04-25T21:08:28.098804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":415,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.520028+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:08:28.520028+0000","last_peered":"2024-04-25T21:08:28.520028+0000","last_clean":"2024-04-25T21:08:28.520028+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:08:28.520028+0000","last_undegraded":"2024-04-25T21:08:28.520028+0000","last_fullsized":"2024-04-25T21:08:28.520028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":378,"num_read_kb":378,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101031+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:08:28.101031+0000","last_peered":"2024-04-25T21:08:28.101031+0000","last_clean":"2024-04-25T21:08:28.101031+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:08:28.101031+0000","last_undegraded":"2024-04-25T21:08:28.101031+0000","last_fullsized":"2024-04-25T21:08:28.101031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358082+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:08:28.358082+0000","last_peered":"2024-04-25T21:08:28.358082+0000","last_clean":"2024-04-25T21:08:28.358082+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:08:28.358082+0000","last_undegraded":"2024-04-25T21:08:28.358082+0000","last_fullsized":"2024-04-25T21:08:28.358082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363081+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:08:28.363081+0000","last_peered":"2024-04-25T21:08:28.363081+0000","last_clean":"2024-04-25T21:08:28.363081+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:08:28.363081+0000","last_undegraded":"2024-04-25T21:08:28.363081+0000","last_fullsized":"2024-04-25T21:08:28.363081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203162+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:08:28.203162+0000","last_peered":"2024-04-25T21:08:28.203162+0000","last_clean":"2024-04-25T21:08:28.203162+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:08:28.203162+0000","last_undegraded":"2024-04-25T21:08:28.203162+0000","last_fullsized":"2024-04-25T21:08:28.203162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100668+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:08:28.100668+0000","last_peered":"2024-04-25T21:08:28.100668+0000","last_clean":"2024-04-25T21:08:28.100668+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:08:28.100668+0000","last_undegraded":"2024-04-25T21:08:28.100668+0000","last_fullsized":"2024-04-25T21:08:28.100668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'39","reported_seq":457,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.539510+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:08:28.539510+0000","last_peered":"2024-04-25T21:08:28.539510+0000","last_clean":"2024-04-25T21:08:28.539510+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:08:28.539510+0000","last_undegraded":"2024-04-25T21:08:28.539510+0000","last_fullsized":"2024-04-25T21:08:28.539510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":389,"num_read_kb":259,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195284+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:08:28.195284+0000","last_peered":"2024-04-25T21:08:28.195284+0000","last_clean":"2024-04-25T21:08:28.195284+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:08:28.195284+0000","last_undegraded":"2024-04-25T21:08:28.195284+0000","last_fullsized":"2024-04-25T21:08:28.195284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101066+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:08:28.101066+0000","last_peered":"2024-04-25T21:08:28.101066+0000","last_clean":"2024-04-25T21:08:28.101066+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:08:28.101066+0000","last_undegraded":"2024-04-25T21:08:28.101066+0000","last_fullsized":"2024-04-25T21:08:28.101066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360236+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:08:28.360236+0000","last_peered":"2024-04-25T21:08:28.360236+0000","last_clean":"2024-04-25T21:08:28.360236+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:08:28.360236+0000","last_undegraded":"2024-04-25T21:08:28.360236+0000","last_fullsized":"2024-04-25T21:08:28.360236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202903+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:08:28.202903+0000","last_peered":"2024-04-25T21:08:28.202903+0000","last_clean":"2024-04-25T21:08:28.202903+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:08:28.202903+0000","last_undegraded":"2024-04-25T21:08:28.202903+0000","last_fullsized":"2024-04-25T21:08:28.202903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357857+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:08:28.357857+0000","last_peered":"2024-04-25T21:08:28.357857+0000","last_clean":"2024-04-25T21:08:28.357857+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:08:28.357857+0000","last_undegraded":"2024-04-25T21:08:28.357857+0000","last_fullsized":"2024-04-25T21:08:28.357857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'6","reported_seq":227,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.507150+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:08:28.507150+0000","last_peered":"2024-04-25T21:08:28.507150+0000","last_clean":"2024-04-25T21:08:28.507150+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:08:28.507150+0000","last_undegraded":"2024-04-25T21:08:28.507150+0000","last_fullsized":"2024-04-25T21:08:28.507150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357601+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:08:28.357601+0000","last_peered":"2024-04-25T21:08:28.357601+0000","last_clean":"2024-04-25T21:08:28.357601+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:08:28.357601+0000","last_undegraded":"2024-04-25T21:08:28.357601+0000","last_fullsized":"2024-04-25T21:08:28.357601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195075+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:08:28.195075+0000","last_peered":"2024-04-25T21:08:28.195075+0000","last_clean":"2024-04-25T21:08:28.195075+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:08:28.195075+0000","last_undegraded":"2024-04-25T21:08:28.195075+0000","last_fullsized":"2024-04-25T21:08:28.195075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202900+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:08:28.202900+0000","last_peered":"2024-04-25T21:08:28.202900+0000","last_clean":"2024-04-25T21:08:28.202900+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:08:28.202900+0000","last_undegraded":"2024-04-25T21:08:28.202900+0000","last_fullsized":"2024-04-25T21:08:28.202900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195478+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:08:28.195478+0000","last_peered":"2024-04-25T21:08:28.195478+0000","last_clean":"2024-04-25T21:08:28.195478+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:08:28.195478+0000","last_undegraded":"2024-04-25T21:08:28.195478+0000","last_fullsized":"2024-04-25T21:08:28.195478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362402+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:08:28.362402+0000","last_peered":"2024-04-25T21:08:28.362402+0000","last_clean":"2024-04-25T21:08:28.362402+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:08:28.362402+0000","last_undegraded":"2024-04-25T21:08:28.362402+0000","last_fullsized":"2024-04-25T21:08:28.362402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098985+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:08:28.098985+0000","last_peered":"2024-04-25T21:08:28.098985+0000","last_clean":"2024-04-25T21:08:28.098985+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:08:28.098985+0000","last_undegraded":"2024-04-25T21:08:28.098985+0000","last_fullsized":"2024-04-25T21:08:28.098985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206100+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:08:28.206100+0000","last_peered":"2024-04-25T21:08:28.206100+0000","last_clean":"2024-04-25T21:08:28.206100+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:08:28.206100+0000","last_undegraded":"2024-04-25T21:08:28.206100+0000","last_fullsized":"2024-04-25T21:08:28.206100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":225,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.506912+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:08:28.506912+0000","last_peered":"2024-04-25T21:08:28.506912+0000","last_clean":"2024-04-25T21:08:28.506912+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:08:28.506912+0000","last_undegraded":"2024-04-25T21:08:28.506912+0000","last_fullsized":"2024-04-25T21:08:28.506912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":191,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205831+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:08:28.205831+0000","last_peered":"2024-04-25T21:08:28.205831+0000","last_clean":"2024-04-25T21:08:28.205831+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:08:28.205831+0000","last_undegraded":"2024-04-25T21:08:28.205831+0000","last_fullsized":"2024-04-25T21:08:28.205831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202984+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:08:28.202984+0000","last_peered":"2024-04-25T21:08:28.202984+0000","last_clean":"2024-04-25T21:08:28.202984+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:08:28.202984+0000","last_undegraded":"2024-04-25T21:08:28.202984+0000","last_fullsized":"2024-04-25T21:08:28.202984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":224,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.364046+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:08:28.364046+0000","last_peered":"2024-04-25T21:08:28.364046+0000","last_clean":"2024-04-25T21:08:28.364046+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:08:28.364046+0000","last_undegraded":"2024-04-25T21:08:28.364046+0000","last_fullsized":"2024-04-25T21:08:28.364046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194859+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:08:28.194859+0000","last_peered":"2024-04-25T21:08:28.194859+0000","last_clean":"2024-04-25T21:08:28.194859+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:08:28.194859+0000","last_undegraded":"2024-04-25T21:08:28.194859+0000","last_fullsized":"2024-04-25T21:08:28.194859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363893+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:08:28.363893+0000","last_peered":"2024-04-25T21:08:28.363893+0000","last_clean":"2024-04-25T21:08:28.363893+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:08:28.363893+0000","last_undegraded":"2024-04-25T21:08:28.363893+0000","last_fullsized":"2024-04-25T21:08:28.363893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":359,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357690+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:08:28.357690+0000","last_peered":"2024-04-25T21:08:28.357690+0000","last_clean":"2024-04-25T21:08:28.357690+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:08:28.357690+0000","last_undegraded":"2024-04-25T21:08:28.357690+0000","last_fullsized":"2024-04-25T21:08:28.357690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359561+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:08:28.359561+0000","last_peered":"2024-04-25T21:08:28.359561+0000","last_clean":"2024-04-25T21:08:28.359561+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:08:28.359561+0000","last_undegraded":"2024-04-25T21:08:28.359561+0000","last_fullsized":"2024-04-25T21:08:28.359561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357841+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:08:28.357841+0000","last_peered":"2024-04-25T21:08:28.357841+0000","last_clean":"2024-04-25T21:08:28.357841+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:08:28.357841+0000","last_undegraded":"2024-04-25T21:08:28.357841+0000","last_fullsized":"2024-04-25T21:08:28.357841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195361+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:08:28.195361+0000","last_peered":"2024-04-25T21:08:28.195361+0000","last_clean":"2024-04-25T21:08:28.195361+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:08:28.195361+0000","last_undegraded":"2024-04-25T21:08:28.195361+0000","last_fullsized":"2024-04-25T21:08:28.195361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360741+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:08:28.360741+0000","last_peered":"2024-04-25T21:08:28.360741+0000","last_clean":"2024-04-25T21:08:28.360741+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:08:28.360741+0000","last_undegraded":"2024-04-25T21:08:28.360741+0000","last_fullsized":"2024-04-25T21:08:28.360741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'51","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.313892+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:08:28.313892+0000","last_peered":"2024-04-25T21:08:28.313892+0000","last_clean":"2024-04-25T21:08:28.313892+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:08:28.313892+0000","last_undegraded":"2024-04-25T21:08:28.313892+0000","last_fullsized":"2024-04-25T21:08:28.313892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195491+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:08:28.195491+0000","last_peered":"2024-04-25T21:08:28.195491+0000","last_clean":"2024-04-25T21:08:28.195491+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:08:28.195491+0000","last_undegraded":"2024-04-25T21:08:28.195491+0000","last_fullsized":"2024-04-25T21:08:28.195491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.364109+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:08:28.364109+0000","last_peered":"2024-04-25T21:08:28.364109+0000","last_clean":"2024-04-25T21:08:28.364109+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:08:28.364109+0000","last_undegraded":"2024-04-25T21:08:28.364109+0000","last_fullsized":"2024-04-25T21:08:28.364109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362959+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:08:28.362959+0000","last_peered":"2024-04-25T21:08:28.362959+0000","last_clean":"2024-04-25T21:08:28.362959+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:08:28.362959+0000","last_undegraded":"2024-04-25T21:08:28.362959+0000","last_fullsized":"2024-04-25T21:08:28.362959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100466+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:08:28.100466+0000","last_peered":"2024-04-25T21:08:28.100466+0000","last_clean":"2024-04-25T21:08:28.100466+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:08:28.100466+0000","last_undegraded":"2024-04-25T21:08:28.100466+0000","last_fullsized":"2024-04-25T21:08:28.100466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195078+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:08:28.195078+0000","last_peered":"2024-04-25T21:08:28.195078+0000","last_clean":"2024-04-25T21:08:28.195078+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:08:28.195078+0000","last_undegraded":"2024-04-25T21:08:28.195078+0000","last_fullsized":"2024-04-25T21:08:28.195078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194864+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:08:28.194864+0000","last_peered":"2024-04-25T21:08:28.194864+0000","last_clean":"2024-04-25T21:08:28.194864+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:08:28.194864+0000","last_undegraded":"2024-04-25T21:08:28.194864+0000","last_fullsized":"2024-04-25T21:08:28.194864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362215+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:08:28.362215+0000","last_peered":"2024-04-25T21:08:28.362215+0000","last_clean":"2024-04-25T21:08:28.362215+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:08:28.362215+0000","last_undegraded":"2024-04-25T21:08:28.362215+0000","last_fullsized":"2024-04-25T21:08:28.362215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.769735+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:08:28.769735+0000","last_peered":"2024-04-25T21:08:28.769735+0000","last_clean":"2024-04-25T21:08:28.769735+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:08:28.769735+0000","last_undegraded":"2024-04-25T21:08:28.769735+0000","last_fullsized":"2024-04-25T21:08:28.769735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195175+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:08:28.195175+0000","last_peered":"2024-04-25T21:08:28.195175+0000","last_clean":"2024-04-25T21:08:28.195175+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:08:28.195175+0000","last_undegraded":"2024-04-25T21:08:28.195175+0000","last_fullsized":"2024-04-25T21:08:28.195175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195252+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:08:28.195252+0000","last_peered":"2024-04-25T21:08:28.195252+0000","last_clean":"2024-04-25T21:08:28.195252+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:08:28.195252+0000","last_undegraded":"2024-04-25T21:08:28.195252+0000","last_fullsized":"2024-04-25T21:08:28.195252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357809+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:08:28.357809+0000","last_peered":"2024-04-25T21:08:28.357809+0000","last_clean":"2024-04-25T21:08:28.357809+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:08:28.357809+0000","last_undegraded":"2024-04-25T21:08:28.357809+0000","last_fullsized":"2024-04-25T21:08:28.357809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.539499+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:08:28.539499+0000","last_peered":"2024-04-25T21:08:28.539499+0000","last_clean":"2024-04-25T21:08:28.539499+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:08:28.539499+0000","last_undegraded":"2024-04-25T21:08:28.539499+0000","last_fullsized":"2024-04-25T21:08:28.539499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358171+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:08:28.358171+0000","last_peered":"2024-04-25T21:08:28.358171+0000","last_clean":"2024-04-25T21:08:28.358171+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:08:28.358171+0000","last_undegraded":"2024-04-25T21:08:28.358171+0000","last_fullsized":"2024-04-25T21:08:28.358171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.311517+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:08:28.311517+0000","last_peered":"2024-04-25T21:08:28.311517+0000","last_clean":"2024-04-25T21:08:28.311517+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:08:28.311517+0000","last_undegraded":"2024-04-25T21:08:28.311517+0000","last_fullsized":"2024-04-25T21:08:28.311517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357893+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:08:28.357893+0000","last_peered":"2024-04-25T21:08:28.357893+0000","last_clean":"2024-04-25T21:08:28.357893+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:08:28.357893+0000","last_undegraded":"2024-04-25T21:08:28.357893+0000","last_fullsized":"2024-04-25T21:08:28.357893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360443+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:08:28.360443+0000","last_peered":"2024-04-25T21:08:28.360443+0000","last_clean":"2024-04-25T21:08:28.360443+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:08:28.360443+0000","last_undegraded":"2024-04-25T21:08:28.360443+0000","last_fullsized":"2024-04-25T21:08:28.360443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358180+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:08:28.358180+0000","last_peered":"2024-04-25T21:08:28.358180+0000","last_clean":"2024-04-25T21:08:28.358180+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:08:28.358180+0000","last_undegraded":"2024-04-25T21:08:28.358180+0000","last_fullsized":"2024-04-25T21:08:28.358180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362227+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:08:28.362227+0000","last_peered":"2024-04-25T21:08:28.362227+0000","last_clean":"2024-04-25T21:08:28.362227+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:08:28.362227+0000","last_undegraded":"2024-04-25T21:08:28.362227+0000","last_fullsized":"2024-04-25T21:08:28.362227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100956+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:08:28.100956+0000","last_peered":"2024-04-25T21:08:28.100956+0000","last_clean":"2024-04-25T21:08:28.100956+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:08:28.100956+0000","last_undegraded":"2024-04-25T21:08:28.100956+0000","last_fullsized":"2024-04-25T21:08:28.100956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100859+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:08:28.100859+0000","last_peered":"2024-04-25T21:08:28.100859+0000","last_clean":"2024-04-25T21:08:28.100859+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:08:28.100859+0000","last_undegraded":"2024-04-25T21:08:28.100859+0000","last_fullsized":"2024-04-25T21:08:28.100859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360781+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:08:28.360781+0000","last_peered":"2024-04-25T21:08:28.360781+0000","last_clean":"2024-04-25T21:08:28.360781+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:08:28.360781+0000","last_undegraded":"2024-04-25T21:08:28.360781+0000","last_fullsized":"2024-04-25T21:08:28.360781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362381+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:08:28.362381+0000","last_peered":"2024-04-25T21:08:28.362381+0000","last_clean":"2024-04-25T21:08:28.362381+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:08:28.362381+0000","last_undegraded":"2024-04-25T21:08:28.362381+0000","last_fullsized":"2024-04-25T21:08:28.362381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101477+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:08:28.101477+0000","last_peered":"2024-04-25T21:08:28.101477+0000","last_clean":"2024-04-25T21:08:28.101477+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:08:28.101477+0000","last_undegraded":"2024-04-25T21:08:28.101477+0000","last_fullsized":"2024-04-25T21:08:28.101477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195449+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:08:28.195449+0000","last_peered":"2024-04-25T21:08:28.195449+0000","last_clean":"2024-04-25T21:08:28.195449+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:08:28.195449+0000","last_undegraded":"2024-04-25T21:08:28.195449+0000","last_fullsized":"2024-04-25T21:08:28.195449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202804+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:08:28.202804+0000","last_peered":"2024-04-25T21:08:28.202804+0000","last_clean":"2024-04-25T21:08:28.202804+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:08:28.202804+0000","last_undegraded":"2024-04-25T21:08:28.202804+0000","last_fullsized":"2024-04-25T21:08:28.202804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194954+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:08:28.194954+0000","last_peered":"2024-04-25T21:08:28.194954+0000","last_clean":"2024-04-25T21:08:28.194954+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:08:28.194954+0000","last_undegraded":"2024-04-25T21:08:28.194954+0000","last_fullsized":"2024-04-25T21:08:28.194954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100689+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:08:28.100689+0000","last_peered":"2024-04-25T21:08:28.100689+0000","last_clean":"2024-04-25T21:08:28.100689+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:08:28.100689+0000","last_undegraded":"2024-04-25T21:08:28.100689+0000","last_fullsized":"2024-04-25T21:08:28.100689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360553+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:08:28.360553+0000","last_peered":"2024-04-25T21:08:28.360553+0000","last_clean":"2024-04-25T21:08:28.360553+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:08:28.360553+0000","last_undegraded":"2024-04-25T21:08:28.360553+0000","last_fullsized":"2024-04-25T21:08:28.360553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362318+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:08:28.362318+0000","last_peered":"2024-04-25T21:08:28.362318+0000","last_clean":"2024-04-25T21:08:28.362318+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:08:28.362318+0000","last_undegraded":"2024-04-25T21:08:28.362318+0000","last_fullsized":"2024-04-25T21:08:28.362318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601039,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2031,"num_read_kb":1132,"num_write":382,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1803117,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383738,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":491622,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645374,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1278912,"data_compressed":44712,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":882564,"data_compressed":22056,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070537,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":882360,"data_compressed":22056,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332174,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":1083558,"data_compressed":38556,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495699,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":489456,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":556843,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":492026,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":557262,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920862,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489317,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":1214054,"data_compressed":44712,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135343,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200201,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:08:34.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:08:34.324 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:34.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:34.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:34.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:34.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:34.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:34.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:34.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:34.987 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi098:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='client.24958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:35.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: Updating smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:08:35.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:08:35.872 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:35 smithi037 ceph-mon[29195]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-04-25T21:08:35.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:35 smithi098 ceph-mon[32569]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-04-25T21:08:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:35 smithi177 ceph-mon[33043]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 183 KiB/s wr, 78 op/s 2024-04-25T21:08:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 -- 172.21.15.37:0/875949771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734406bbb0 msgr2=0x7f734410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 --2- 172.21.15.37:0/875949771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734406bbb0 0x7f734410f930 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b0a0 tx=0x7f733c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 -- 172.21.15.37:0/875949771 shutdown_connections 2024-04-25T21:08:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 --2- 172.21.15.37:0/875949771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734406bbb0 0x7f734410f930 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 --2- 172.21.15.37:0/875949771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7344108a60 0x7f7344108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 --2- 172.21.15.37:0/875949771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406b1f0 0x7f734406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 -- 172.21.15.37:0/875949771 >> 172.21.15.37:0/875949771 conn(0x7f73440789e0 msgr2=0x7f7344078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 -- 172.21.15.37:0/875949771 shutdown_connections 2024-04-25T21:08:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.114+0000 7f7348ea3700 1 -- 172.21.15.37:0/875949771 wait complete. 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 Processor -- start 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 -- start start 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 0x7f73441990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406bbb0 0x7f7344199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7344112ca0 con 0x7f7344108a60 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7344112b20 con 0x7f734406bbb0 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7348ea3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7344112e20 con 0x7f734406b1f0 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50132/0 (socket says 172.21.15.37:50132) 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 -- 172.21.15.37:0/2405721534 learned_addr learned my addr 172.21.15.37:0/2405721534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f734259c700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 0x7f73441990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7341d9b700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406bbb0 0x7f7344199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 -- 172.21.15.37:0/2405721534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 msgr2=0x7f73441990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 0x7f73441990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 -- 172.21.15.37:0/2405721534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406bbb0 msgr2=0x7f7344199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406bbb0 0x7f7344199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f7342d9d700 1 -- 172.21.15.37:0/2405721534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419e090 con 0x7f7344108a60 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.115+0000 7f734259c700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 0x7f73441990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f7342d9d700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b070 tx=0x7f733c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c021070 con 0x7f7344108a60 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c007c80 con 0x7f7344108a60 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c005440 con 0x7f7344108a60 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419e320 con 0x7f7344108a60 2024-04-25T21:08:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.116+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73441a5c00 con 0x7f7344108a60 2024-04-25T21:08:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.117+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344069730 con 0x7f7344108a60 2024-04-25T21:08:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.118+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f733c024070 con 0x7f7344108a60 2024-04-25T21:08:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.119+0000 7f73337fe700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 0x7f732c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.119+0000 7f734259c700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 0x7f732c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.119+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f733c00f020 con 0x7f7344108a60 2024-04-25T21:08:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.119+0000 7f734259c700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 0x7f732c0a6410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f733400a5e0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:36.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.124+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f733c02b020 con 0x7f7344108a60 2024-04-25T21:08:36.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.330+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7344062e80 con 0x7f732c0a3f50 2024-04-25T21:08:36.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.337+0000 7f73337fe700 1 -- 172.21.15.37:0/2405721534 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258091 (secure 0 0 0) 0x7f7344062e80 con 0x7f732c0a3f50 2024-04-25T21:08:36.341 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 msgr2=0x7f732c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 0x7f732c0a6410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7334005fd0 tx=0x7f733400a5e0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 msgr2=0x7f734419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b070 tx=0x7f733c00ce50 comp rx=0 tx=0).stop 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 shutdown_connections 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f732c0a3f50 0x7f732c0a6410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7344108a60 0x7f734419d950 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734406b1f0 0x7f73441990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 --2- 172.21.15.37:0/2405721534 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f734406bbb0 0x7f7344199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 >> 172.21.15.37:0/2405721534 conn(0x7f73440789e0 msgr2=0x7f73440ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:36.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 shutdown_connections 2024-04-25T21:08:36.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:36.340+0000 7f7348ea3700 1 -- 172.21.15.37:0/2405721534 wait complete. 2024-04-25T21:08:36.344 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:08:37.163 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-04-25T21:08:36.114485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391575,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2574,"num_read_kb":1722,"num_write":718,"num_write_kb":5331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1042,"ondisk_log_size":1042,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365868,"kb_used_data":12516,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575956,"statfs":{"total":1151940427776,"available":1151565778944,"internally_reserved":0,"allocated":12816384,"data_stored":10033437,"data_compressed":245331,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":407,"num_read_kb":471,"num_write":153,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.988957"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362557+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:08:28.362557+0000","last_peered":"2024-04-25T21:08:28.362557+0000","last_clean":"2024-04-25T21:08:28.362557+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:08:28.362557+0000","last_undegraded":"2024-04-25T21:08:28.362557+0000","last_fullsized":"2024-04-25T21:08:28.362557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362924+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:08:28.362924+0000","last_peered":"2024-04-25T21:08:28.362924+0000","last_clean":"2024-04-25T21:08:28.362924+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:08:28.362924+0000","last_undegraded":"2024-04-25T21:08:28.362924+0000","last_fullsized":"2024-04-25T21:08:28.362924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363987+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:08:28.363987+0000","last_peered":"2024-04-25T21:08:28.363987+0000","last_clean":"2024-04-25T21:08:28.363987+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:08:28.363987+0000","last_undegraded":"2024-04-25T21:08:28.363987+0000","last_fullsized":"2024-04-25T21:08:28.363987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195181+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:08:28.195181+0000","last_peered":"2024-04-25T21:08:28.195181+0000","last_clean":"2024-04-25T21:08:28.195181+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:08:28.195181+0000","last_undegraded":"2024-04-25T21:08:28.195181+0000","last_fullsized":"2024-04-25T21:08:28.195181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205898+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:08:28.205898+0000","last_peered":"2024-04-25T21:08:28.205898+0000","last_clean":"2024-04-25T21:08:28.205898+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:08:28.205898+0000","last_undegraded":"2024-04-25T21:08:28.205898+0000","last_fullsized":"2024-04-25T21:08:28.205898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357638+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:08:28.357638+0000","last_peered":"2024-04-25T21:08:28.357638+0000","last_clean":"2024-04-25T21:08:28.357638+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:08:28.357638+0000","last_undegraded":"2024-04-25T21:08:28.357638+0000","last_fullsized":"2024-04-25T21:08:28.357638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194847+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:08:28.194847+0000","last_peered":"2024-04-25T21:08:28.194847+0000","last_clean":"2024-04-25T21:08:28.194847+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:08:28.194847+0000","last_undegraded":"2024-04-25T21:08:28.194847+0000","last_fullsized":"2024-04-25T21:08:28.194847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101029+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:08:28.101029+0000","last_peered":"2024-04-25T21:08:28.101029+0000","last_clean":"2024-04-25T21:08:28.101029+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:08:28.101029+0000","last_undegraded":"2024-04-25T21:08:28.101029+0000","last_fullsized":"2024-04-25T21:08:28.101029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206257+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:08:28.206257+0000","last_peered":"2024-04-25T21:08:28.206257+0000","last_clean":"2024-04-25T21:08:28.206257+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:08:28.206257+0000","last_undegraded":"2024-04-25T21:08:28.206257+0000","last_fullsized":"2024-04-25T21:08:28.206257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358006+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:08:28.358006+0000","last_peered":"2024-04-25T21:08:28.358006+0000","last_clean":"2024-04-25T21:08:28.358006+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:08:28.358006+0000","last_undegraded":"2024-04-25T21:08:28.358006+0000","last_fullsized":"2024-04-25T21:08:28.358006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363955+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:08:28.363955+0000","last_peered":"2024-04-25T21:08:28.363955+0000","last_clean":"2024-04-25T21:08:28.363955+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:08:28.363955+0000","last_undegraded":"2024-04-25T21:08:28.363955+0000","last_fullsized":"2024-04-25T21:08:28.363955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360589+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:08:28.360589+0000","last_peered":"2024-04-25T21:08:28.360589+0000","last_clean":"2024-04-25T21:08:28.360589+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:08:28.360589+0000","last_undegraded":"2024-04-25T21:08:28.360589+0000","last_fullsized":"2024-04-25T21:08:28.360589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362450+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:08:28.362450+0000","last_peered":"2024-04-25T21:08:28.362450+0000","last_clean":"2024-04-25T21:08:28.362450+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:08:28.362450+0000","last_undegraded":"2024-04-25T21:08:28.362450+0000","last_fullsized":"2024-04-25T21:08:28.362450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362235+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:08:28.362235+0000","last_peered":"2024-04-25T21:08:28.362235+0000","last_clean":"2024-04-25T21:08:28.362235+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:08:28.362235+0000","last_undegraded":"2024-04-25T21:08:28.362235+0000","last_fullsized":"2024-04-25T21:08:28.362235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100450+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:08:28.100450+0000","last_peered":"2024-04-25T21:08:28.100450+0000","last_clean":"2024-04-25T21:08:28.100450+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:08:28.100450+0000","last_undegraded":"2024-04-25T21:08:28.100450+0000","last_fullsized":"2024-04-25T21:08:28.100450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362697+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:08:28.362697+0000","last_peered":"2024-04-25T21:08:28.362697+0000","last_clean":"2024-04-25T21:08:28.362697+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:08:28.362697+0000","last_undegraded":"2024-04-25T21:08:28.362697+0000","last_fullsized":"2024-04-25T21:08:28.362697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.099215+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:08:28.099215+0000","last_peered":"2024-04-25T21:08:28.099215+0000","last_clean":"2024-04-25T21:08:28.099215+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:08:28.099215+0000","last_undegraded":"2024-04-25T21:08:28.099215+0000","last_fullsized":"2024-04-25T21:08:28.099215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101330+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:08:28.101330+0000","last_peered":"2024-04-25T21:08:28.101330+0000","last_clean":"2024-04-25T21:08:28.101330+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:08:28.101330+0000","last_undegraded":"2024-04-25T21:08:28.101330+0000","last_fullsized":"2024-04-25T21:08:28.101330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.511922+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:08:32.511922+0000","last_peered":"2024-04-25T21:08:32.511922+0000","last_clean":"2024-04-25T21:08:32.511922+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:08:32.511922+0000","last_undegraded":"2024-04-25T21:08:32.511922+0000","last_fullsized":"2024-04-25T21:08:32.511922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195064+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:08:28.195064+0000","last_peered":"2024-04-25T21:08:28.195064+0000","last_clean":"2024-04-25T21:08:28.195064+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:08:28.195064+0000","last_undegraded":"2024-04-25T21:08:28.195064+0000","last_fullsized":"2024-04-25T21:08:28.195064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195494+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:08:28.195494+0000","last_peered":"2024-04-25T21:08:28.195494+0000","last_clean":"2024-04-25T21:08:28.195494+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:08:28.195494+0000","last_undegraded":"2024-04-25T21:08:28.195494+0000","last_fullsized":"2024-04-25T21:08:28.195494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363253+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:08:28.363253+0000","last_peered":"2024-04-25T21:08:28.363253+0000","last_clean":"2024-04-25T21:08:28.363253+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:08:28.363253+0000","last_undegraded":"2024-04-25T21:08:28.363253+0000","last_fullsized":"2024-04-25T21:08:28.363253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195601+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:08:28.195601+0000","last_peered":"2024-04-25T21:08:28.195601+0000","last_clean":"2024-04-25T21:08:28.195601+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:08:28.195601+0000","last_undegraded":"2024-04-25T21:08:28.195601+0000","last_fullsized":"2024-04-25T21:08:28.195601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100566+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:08:28.100566+0000","last_peered":"2024-04-25T21:08:28.100566+0000","last_clean":"2024-04-25T21:08:28.100566+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:08:28.100566+0000","last_undegraded":"2024-04-25T21:08:28.100566+0000","last_fullsized":"2024-04-25T21:08:28.100566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362183+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:08:28.362183+0000","last_peered":"2024-04-25T21:08:28.362183+0000","last_clean":"2024-04-25T21:08:28.362183+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:08:28.362183+0000","last_undegraded":"2024-04-25T21:08:28.362183+0000","last_fullsized":"2024-04-25T21:08:28.362183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203110+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:08:28.203110+0000","last_peered":"2024-04-25T21:08:28.203110+0000","last_clean":"2024-04-25T21:08:28.203110+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:08:28.203110+0000","last_undegraded":"2024-04-25T21:08:28.203110+0000","last_fullsized":"2024-04-25T21:08:28.203110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":236,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.527332+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:08:32.527332+0000","last_peered":"2024-04-25T21:08:32.527332+0000","last_clean":"2024-04-25T21:08:32.527332+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:08:32.527332+0000","last_undegraded":"2024-04-25T21:08:32.527332+0000","last_fullsized":"2024-04-25T21:08:32.527332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195157+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:08:28.195157+0000","last_peered":"2024-04-25T21:08:28.195157+0000","last_clean":"2024-04-25T21:08:28.195157+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:08:28.195157+0000","last_undegraded":"2024-04-25T21:08:28.195157+0000","last_fullsized":"2024-04-25T21:08:28.195157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205818+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:08:28.205818+0000","last_peered":"2024-04-25T21:08:28.205818+0000","last_clean":"2024-04-25T21:08:28.205818+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:08:28.205818+0000","last_undegraded":"2024-04-25T21:08:28.205818+0000","last_fullsized":"2024-04-25T21:08:28.205818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203072+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:08:28.203072+0000","last_peered":"2024-04-25T21:08:28.203072+0000","last_clean":"2024-04-25T21:08:28.203072+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:08:28.203072+0000","last_undegraded":"2024-04-25T21:08:28.203072+0000","last_fullsized":"2024-04-25T21:08:28.203072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359792+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:08:28.359792+0000","last_peered":"2024-04-25T21:08:28.359792+0000","last_clean":"2024-04-25T21:08:28.359792+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:08:28.359792+0000","last_undegraded":"2024-04-25T21:08:28.359792+0000","last_fullsized":"2024-04-25T21:08:28.359792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194771+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:08:28.194771+0000","last_peered":"2024-04-25T21:08:28.194771+0000","last_clean":"2024-04-25T21:08:28.194771+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:08:28.194771+0000","last_undegraded":"2024-04-25T21:08:28.194771+0000","last_fullsized":"2024-04-25T21:08:28.194771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202732+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:08:28.202732+0000","last_peered":"2024-04-25T21:08:28.202732+0000","last_clean":"2024-04-25T21:08:28.202732+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:08:28.202732+0000","last_undegraded":"2024-04-25T21:08:28.202732+0000","last_fullsized":"2024-04-25T21:08:28.202732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098804+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:08:28.098804+0000","last_peered":"2024-04-25T21:08:28.098804+0000","last_clean":"2024-04-25T21:08:28.098804+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:08:28.098804+0000","last_undegraded":"2024-04-25T21:08:28.098804+0000","last_fullsized":"2024-04-25T21:08:28.098804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":425,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:33.528301+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:08:33.528301+0000","last_peered":"2024-04-25T21:08:33.528301+0000","last_clean":"2024-04-25T21:08:33.528301+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:08:33.528301+0000","last_undegraded":"2024-04-25T21:08:33.528301+0000","last_fullsized":"2024-04-25T21:08:33.528301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":388,"num_read_kb":388,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101031+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:08:28.101031+0000","last_peered":"2024-04-25T21:08:28.101031+0000","last_clean":"2024-04-25T21:08:28.101031+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:08:28.101031+0000","last_undegraded":"2024-04-25T21:08:28.101031+0000","last_fullsized":"2024-04-25T21:08:28.101031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358082+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:08:28.358082+0000","last_peered":"2024-04-25T21:08:28.358082+0000","last_clean":"2024-04-25T21:08:28.358082+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:08:28.358082+0000","last_undegraded":"2024-04-25T21:08:28.358082+0000","last_fullsized":"2024-04-25T21:08:28.358082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363081+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:08:28.363081+0000","last_peered":"2024-04-25T21:08:28.363081+0000","last_clean":"2024-04-25T21:08:28.363081+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:08:28.363081+0000","last_undegraded":"2024-04-25T21:08:28.363081+0000","last_fullsized":"2024-04-25T21:08:28.363081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203162+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:08:28.203162+0000","last_peered":"2024-04-25T21:08:28.203162+0000","last_clean":"2024-04-25T21:08:28.203162+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:08:28.203162+0000","last_undegraded":"2024-04-25T21:08:28.203162+0000","last_fullsized":"2024-04-25T21:08:28.203162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100668+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:08:28.100668+0000","last_peered":"2024-04-25T21:08:28.100668+0000","last_clean":"2024-04-25T21:08:28.100668+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:08:28.100668+0000","last_undegraded":"2024-04-25T21:08:28.100668+0000","last_fullsized":"2024-04-25T21:08:28.100668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":471,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:33.546339+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:08:33.546339+0000","last_peered":"2024-04-25T21:08:33.546339+0000","last_clean":"2024-04-25T21:08:33.546339+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:08:33.546339+0000","last_undegraded":"2024-04-25T21:08:33.546339+0000","last_fullsized":"2024-04-25T21:08:33.546339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":268,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195284+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:08:28.195284+0000","last_peered":"2024-04-25T21:08:28.195284+0000","last_clean":"2024-04-25T21:08:28.195284+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:08:28.195284+0000","last_undegraded":"2024-04-25T21:08:28.195284+0000","last_fullsized":"2024-04-25T21:08:28.195284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101066+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:08:28.101066+0000","last_peered":"2024-04-25T21:08:28.101066+0000","last_clean":"2024-04-25T21:08:28.101066+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:08:28.101066+0000","last_undegraded":"2024-04-25T21:08:28.101066+0000","last_fullsized":"2024-04-25T21:08:28.101066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360236+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:08:28.360236+0000","last_peered":"2024-04-25T21:08:28.360236+0000","last_clean":"2024-04-25T21:08:28.360236+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:08:28.360236+0000","last_undegraded":"2024-04-25T21:08:28.360236+0000","last_fullsized":"2024-04-25T21:08:28.360236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202903+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:08:28.202903+0000","last_peered":"2024-04-25T21:08:28.202903+0000","last_clean":"2024-04-25T21:08:28.202903+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:08:28.202903+0000","last_undegraded":"2024-04-25T21:08:28.202903+0000","last_fullsized":"2024-04-25T21:08:28.202903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357857+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:08:28.357857+0000","last_peered":"2024-04-25T21:08:28.357857+0000","last_clean":"2024-04-25T21:08:28.357857+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:08:28.357857+0000","last_undegraded":"2024-04-25T21:08:28.357857+0000","last_fullsized":"2024-04-25T21:08:28.357857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512998+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:08:32.512998+0000","last_peered":"2024-04-25T21:08:32.512998+0000","last_clean":"2024-04-25T21:08:32.512998+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:08:32.512998+0000","last_undegraded":"2024-04-25T21:08:32.512998+0000","last_fullsized":"2024-04-25T21:08:32.512998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357601+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:08:28.357601+0000","last_peered":"2024-04-25T21:08:28.357601+0000","last_clean":"2024-04-25T21:08:28.357601+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:08:28.357601+0000","last_undegraded":"2024-04-25T21:08:28.357601+0000","last_fullsized":"2024-04-25T21:08:28.357601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195075+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:08:28.195075+0000","last_peered":"2024-04-25T21:08:28.195075+0000","last_clean":"2024-04-25T21:08:28.195075+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:08:28.195075+0000","last_undegraded":"2024-04-25T21:08:28.195075+0000","last_fullsized":"2024-04-25T21:08:28.195075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202900+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:08:28.202900+0000","last_peered":"2024-04-25T21:08:28.202900+0000","last_clean":"2024-04-25T21:08:28.202900+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:08:28.202900+0000","last_undegraded":"2024-04-25T21:08:28.202900+0000","last_fullsized":"2024-04-25T21:08:28.202900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195524+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:08:28.195524+0000","last_peered":"2024-04-25T21:08:28.195524+0000","last_clean":"2024-04-25T21:08:28.195524+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:08:28.195524+0000","last_undegraded":"2024-04-25T21:08:28.195524+0000","last_fullsized":"2024-04-25T21:08:28.195524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362402+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:08:28.362402+0000","last_peered":"2024-04-25T21:08:28.362402+0000","last_clean":"2024-04-25T21:08:28.362402+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:08:28.362402+0000","last_undegraded":"2024-04-25T21:08:28.362402+0000","last_fullsized":"2024-04-25T21:08:28.362402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098985+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:08:28.098985+0000","last_peered":"2024-04-25T21:08:28.098985+0000","last_clean":"2024-04-25T21:08:28.098985+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:08:28.098985+0000","last_undegraded":"2024-04-25T21:08:28.098985+0000","last_fullsized":"2024-04-25T21:08:28.098985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206100+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:08:28.206100+0000","last_peered":"2024-04-25T21:08:28.206100+0000","last_clean":"2024-04-25T21:08:28.206100+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:08:28.206100+0000","last_undegraded":"2024-04-25T21:08:28.206100+0000","last_fullsized":"2024-04-25T21:08:28.206100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512665+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:08:32.512665+0000","last_peered":"2024-04-25T21:08:32.512665+0000","last_clean":"2024-04-25T21:08:32.512665+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:08:32.512665+0000","last_undegraded":"2024-04-25T21:08:32.512665+0000","last_fullsized":"2024-04-25T21:08:32.512665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205831+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:08:28.205831+0000","last_peered":"2024-04-25T21:08:28.205831+0000","last_clean":"2024-04-25T21:08:28.205831+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:08:28.205831+0000","last_undegraded":"2024-04-25T21:08:28.205831+0000","last_fullsized":"2024-04-25T21:08:28.205831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202984+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:08:28.202984+0000","last_peered":"2024-04-25T21:08:28.202984+0000","last_clean":"2024-04-25T21:08:28.202984+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:08:28.202984+0000","last_undegraded":"2024-04-25T21:08:28.202984+0000","last_fullsized":"2024-04-25T21:08:28.202984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512521+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:08:32.512521+0000","last_peered":"2024-04-25T21:08:32.512521+0000","last_clean":"2024-04-25T21:08:32.512521+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:08:32.512521+0000","last_undegraded":"2024-04-25T21:08:32.512521+0000","last_fullsized":"2024-04-25T21:08:32.512521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194859+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:08:28.194859+0000","last_peered":"2024-04-25T21:08:28.194859+0000","last_clean":"2024-04-25T21:08:28.194859+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:08:28.194859+0000","last_undegraded":"2024-04-25T21:08:28.194859+0000","last_fullsized":"2024-04-25T21:08:28.194859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363893+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:08:28.363893+0000","last_peered":"2024-04-25T21:08:28.363893+0000","last_clean":"2024-04-25T21:08:28.363893+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:08:28.363893+0000","last_undegraded":"2024-04-25T21:08:28.363893+0000","last_fullsized":"2024-04-25T21:08:28.363893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":403,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.208758+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:08:30.208758+0000","last_peered":"2024-04-25T21:08:30.208758+0000","last_clean":"2024-04-25T21:08:30.208758+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:08:30.208758+0000","last_undegraded":"2024-04-25T21:08:30.208758+0000","last_fullsized":"2024-04-25T21:08:30.208758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359561+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:08:28.359561+0000","last_peered":"2024-04-25T21:08:28.359561+0000","last_clean":"2024-04-25T21:08:28.359561+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:08:28.359561+0000","last_undegraded":"2024-04-25T21:08:28.359561+0000","last_fullsized":"2024-04-25T21:08:28.359561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357841+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:08:28.357841+0000","last_peered":"2024-04-25T21:08:28.357841+0000","last_clean":"2024-04-25T21:08:28.357841+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:08:28.357841+0000","last_undegraded":"2024-04-25T21:08:28.357841+0000","last_fullsized":"2024-04-25T21:08:28.357841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195361+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:08:28.195361+0000","last_peered":"2024-04-25T21:08:28.195361+0000","last_clean":"2024-04-25T21:08:28.195361+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:08:28.195361+0000","last_undegraded":"2024-04-25T21:08:28.195361+0000","last_fullsized":"2024-04-25T21:08:28.195361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360741+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:08:28.360741+0000","last_peered":"2024-04-25T21:08:28.360741+0000","last_clean":"2024-04-25T21:08:28.360741+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:08:28.360741+0000","last_undegraded":"2024-04-25T21:08:28.360741+0000","last_fullsized":"2024-04-25T21:08:28.360741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.727113+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:08:30.727113+0000","last_peered":"2024-04-25T21:08:30.727113+0000","last_clean":"2024-04-25T21:08:30.727113+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:08:30.727113+0000","last_undegraded":"2024-04-25T21:08:30.727113+0000","last_fullsized":"2024-04-25T21:08:30.727113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195491+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:08:28.195491+0000","last_peered":"2024-04-25T21:08:28.195491+0000","last_clean":"2024-04-25T21:08:28.195491+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:08:28.195491+0000","last_undegraded":"2024-04-25T21:08:28.195491+0000","last_fullsized":"2024-04-25T21:08:28.195491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.364109+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:08:28.364109+0000","last_peered":"2024-04-25T21:08:28.364109+0000","last_clean":"2024-04-25T21:08:28.364109+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:08:28.364109+0000","last_undegraded":"2024-04-25T21:08:28.364109+0000","last_fullsized":"2024-04-25T21:08:28.364109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362959+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:08:28.362959+0000","last_peered":"2024-04-25T21:08:28.362959+0000","last_clean":"2024-04-25T21:08:28.362959+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:08:28.362959+0000","last_undegraded":"2024-04-25T21:08:28.362959+0000","last_fullsized":"2024-04-25T21:08:28.362959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100466+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:08:28.100466+0000","last_peered":"2024-04-25T21:08:28.100466+0000","last_clean":"2024-04-25T21:08:28.100466+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:08:28.100466+0000","last_undegraded":"2024-04-25T21:08:28.100466+0000","last_fullsized":"2024-04-25T21:08:28.100466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195078+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:08:28.195078+0000","last_peered":"2024-04-25T21:08:28.195078+0000","last_clean":"2024-04-25T21:08:28.195078+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:08:28.195078+0000","last_undegraded":"2024-04-25T21:08:28.195078+0000","last_fullsized":"2024-04-25T21:08:28.195078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194864+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:08:28.194864+0000","last_peered":"2024-04-25T21:08:28.194864+0000","last_clean":"2024-04-25T21:08:28.194864+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:08:28.194864+0000","last_undegraded":"2024-04-25T21:08:28.194864+0000","last_fullsized":"2024-04-25T21:08:28.194864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362215+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:08:28.362215+0000","last_peered":"2024-04-25T21:08:28.362215+0000","last_clean":"2024-04-25T21:08:28.362215+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:08:28.362215+0000","last_undegraded":"2024-04-25T21:08:28.362215+0000","last_fullsized":"2024-04-25T21:08:28.362215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.769735+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:08:28.769735+0000","last_peered":"2024-04-25T21:08:28.769735+0000","last_clean":"2024-04-25T21:08:28.769735+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:08:28.769735+0000","last_undegraded":"2024-04-25T21:08:28.769735+0000","last_fullsized":"2024-04-25T21:08:28.769735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195175+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:08:28.195175+0000","last_peered":"2024-04-25T21:08:28.195175+0000","last_clean":"2024-04-25T21:08:28.195175+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:08:28.195175+0000","last_undegraded":"2024-04-25T21:08:28.195175+0000","last_fullsized":"2024-04-25T21:08:28.195175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195252+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:08:28.195252+0000","last_peered":"2024-04-25T21:08:28.195252+0000","last_clean":"2024-04-25T21:08:28.195252+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:08:28.195252+0000","last_undegraded":"2024-04-25T21:08:28.195252+0000","last_fullsized":"2024-04-25T21:08:28.195252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357809+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:08:28.357809+0000","last_peered":"2024-04-25T21:08:28.357809+0000","last_clean":"2024-04-25T21:08:28.357809+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:08:28.357809+0000","last_undegraded":"2024-04-25T21:08:28.357809+0000","last_fullsized":"2024-04-25T21:08:28.357809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":239,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.545655+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:08:32.545655+0000","last_peered":"2024-04-25T21:08:32.545655+0000","last_clean":"2024-04-25T21:08:32.545655+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:08:32.545655+0000","last_undegraded":"2024-04-25T21:08:32.545655+0000","last_fullsized":"2024-04-25T21:08:32.545655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358171+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:08:28.358171+0000","last_peered":"2024-04-25T21:08:28.358171+0000","last_clean":"2024-04-25T21:08:28.358171+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:08:28.358171+0000","last_undegraded":"2024-04-25T21:08:28.358171+0000","last_fullsized":"2024-04-25T21:08:28.358171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.369314+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:08:30.369314+0000","last_peered":"2024-04-25T21:08:30.369314+0000","last_clean":"2024-04-25T21:08:30.369314+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:08:30.369314+0000","last_undegraded":"2024-04-25T21:08:30.369314+0000","last_fullsized":"2024-04-25T21:08:30.369314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357893+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:08:28.357893+0000","last_peered":"2024-04-25T21:08:28.357893+0000","last_clean":"2024-04-25T21:08:28.357893+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:08:28.357893+0000","last_undegraded":"2024-04-25T21:08:28.357893+0000","last_fullsized":"2024-04-25T21:08:28.357893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360443+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:08:28.360443+0000","last_peered":"2024-04-25T21:08:28.360443+0000","last_clean":"2024-04-25T21:08:28.360443+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:08:28.360443+0000","last_undegraded":"2024-04-25T21:08:28.360443+0000","last_fullsized":"2024-04-25T21:08:28.360443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358180+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:08:28.358180+0000","last_peered":"2024-04-25T21:08:28.358180+0000","last_clean":"2024-04-25T21:08:28.358180+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:08:28.358180+0000","last_undegraded":"2024-04-25T21:08:28.358180+0000","last_fullsized":"2024-04-25T21:08:28.358180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362227+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:08:28.362227+0000","last_peered":"2024-04-25T21:08:28.362227+0000","last_clean":"2024-04-25T21:08:28.362227+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:08:28.362227+0000","last_undegraded":"2024-04-25T21:08:28.362227+0000","last_fullsized":"2024-04-25T21:08:28.362227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100956+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:08:28.100956+0000","last_peered":"2024-04-25T21:08:28.100956+0000","last_clean":"2024-04-25T21:08:28.100956+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:08:28.100956+0000","last_undegraded":"2024-04-25T21:08:28.100956+0000","last_fullsized":"2024-04-25T21:08:28.100956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100859+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:08:28.100859+0000","last_peered":"2024-04-25T21:08:28.100859+0000","last_clean":"2024-04-25T21:08:28.100859+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:08:28.100859+0000","last_undegraded":"2024-04-25T21:08:28.100859+0000","last_fullsized":"2024-04-25T21:08:28.100859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360781+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:08:28.360781+0000","last_peered":"2024-04-25T21:08:28.360781+0000","last_clean":"2024-04-25T21:08:28.360781+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:08:28.360781+0000","last_undegraded":"2024-04-25T21:08:28.360781+0000","last_fullsized":"2024-04-25T21:08:28.360781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362381+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:08:28.362381+0000","last_peered":"2024-04-25T21:08:28.362381+0000","last_clean":"2024-04-25T21:08:28.362381+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:08:28.362381+0000","last_undegraded":"2024-04-25T21:08:28.362381+0000","last_fullsized":"2024-04-25T21:08:28.362381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101477+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:08:28.101477+0000","last_peered":"2024-04-25T21:08:28.101477+0000","last_clean":"2024-04-25T21:08:28.101477+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:08:28.101477+0000","last_undegraded":"2024-04-25T21:08:28.101477+0000","last_fullsized":"2024-04-25T21:08:28.101477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195449+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:08:28.195449+0000","last_peered":"2024-04-25T21:08:28.195449+0000","last_clean":"2024-04-25T21:08:28.195449+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:08:28.195449+0000","last_undegraded":"2024-04-25T21:08:28.195449+0000","last_fullsized":"2024-04-25T21:08:28.195449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202804+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:08:28.202804+0000","last_peered":"2024-04-25T21:08:28.202804+0000","last_clean":"2024-04-25T21:08:28.202804+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:08:28.202804+0000","last_undegraded":"2024-04-25T21:08:28.202804+0000","last_fullsized":"2024-04-25T21:08:28.202804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194954+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:08:28.194954+0000","last_peered":"2024-04-25T21:08:28.194954+0000","last_clean":"2024-04-25T21:08:28.194954+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:08:28.194954+0000","last_undegraded":"2024-04-25T21:08:28.194954+0000","last_fullsized":"2024-04-25T21:08:28.194954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100689+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:08:28.100689+0000","last_peered":"2024-04-25T21:08:28.100689+0000","last_clean":"2024-04-25T21:08:28.100689+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:08:28.100689+0000","last_undegraded":"2024-04-25T21:08:28.100689+0000","last_fullsized":"2024-04-25T21:08:28.100689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360553+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:08:28.360553+0000","last_peered":"2024-04-25T21:08:28.360553+0000","last_clean":"2024-04-25T21:08:28.360553+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:08:28.360553+0000","last_undegraded":"2024-04-25T21:08:28.360553+0000","last_fullsized":"2024-04-25T21:08:28.360553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362318+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:08:28.362318+0000","last_peered":"2024-04-25T21:08:28.362318+0000","last_clean":"2024-04-25T21:08:28.362318+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:08:28.362318+0000","last_undegraded":"2024-04-25T21:08:28.362318+0000","last_fullsized":"2024-04-25T21:08:28.362318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800973,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2337,"num_read_kb":1392,"num_write":515,"num_write_kb":2391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2402919,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":491622,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":1478846,"data_compressed":53758,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":882564,"data_compressed":22199,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":882360,"data_compressed":22199,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332175,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083949,"data_compressed":38556,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":489456,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":559754,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":492026,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":562128,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1210338,"data_compressed":37401,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489708,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757338,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1410686,"data_compressed":53758,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331975,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400135,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:08:37.168 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T21:08:37.168 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T21:08:37.426 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:37.831 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:37 smithi037 ceph-mon[29195]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-25T21:08:37.831 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:37 smithi037 ceph-mon[29195]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:37 smithi098 ceph-mon[32569]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-25T21:08:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:37 smithi098 ceph-mon[32569]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:37 smithi177 ceph-mon[33043]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-25T21:08:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:37 smithi177 ceph-mon[33043]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 -- 172.21.15.37:0/4056451780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 msgr2=0x7fd2a01118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4056451780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a01118e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b0a0 tx=0x7fd29800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 -- 172.21.15.37:0/4056451780 shutdown_connections 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4056451780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a01118e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4056451780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006a750 0x7fd2a006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4056451780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006b070 0x7fd2a010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 -- 172.21.15.37:0/4056451780 >> 172.21.15.37:0/4056451780 conn(0x7fd2a00788f0 msgr2=0x7fd2a0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 -- 172.21.15.37:0/4056451780 shutdown_connections 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.197+0000 7fd2a59af700 1 -- 172.21.15.37:0/4056451780 wait complete. 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 Processor -- start 2024-04-25T21:08:39.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 -- start start 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006a750 0x7fd2a019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 0x7fd2a019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0114e10 con 0x7fd2a010f4f0 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0114c90 con 0x7fd2a006b070 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd2a59af700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a0114f90 con 0x7fd2a006a750 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd29f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd29effd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006a750 0x7fd2a019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd29f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36752/0 (socket says 172.21.15.37:36752) 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.198+0000 7fd29f7fe700 1 -- 172.21.15.37:0/4045553176 learned_addr learned my addr 172.21.15.37:0/4045553176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29e7fc700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 0x7fd2a019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 -- 172.21.15.37:0/4045553176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006a750 msgr2=0x7fd2a019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006a750 0x7fd2a019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 -- 172.21.15.37:0/4045553176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 msgr2=0x7fd2a019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 0x7fd2a019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 -- 172.21.15.37:0/4045553176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a01a0280 con 0x7fd2a010f4f0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29e7fc700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 0x7fd2a019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd29f7fe700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd298006070 tx=0x7fd298004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298021070 con 0x7fd2a010f4f0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd298007c80 con 0x7fd2a010f4f0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a01a0510 con 0x7fd2a010f4f0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2980047b0 con 0x7fd2a010f4f0 2024-04-25T21:08:39.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.199+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a01a7df0 con 0x7fd2a010f4f0 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.200+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a0069730 con 0x7fd2a010f4f0 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.201+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd298003a00 con 0x7fd2a010f4f0 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.202+0000 7fd2a49ad700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 0x7fd2880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.202+0000 7fd29effd700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 0x7fd2880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.202+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd29800f020 con 0x7fd2a010f4f0 2024-04-25T21:08:39.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.203+0000 7fd29effd700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 0x7fd2880a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd29000ac50 tx=0x7fd29000a380 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:39.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.206+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd29802a020 con 0x7fd2a010f4f0 2024-04-25T21:08:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.409+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2a019c5d0 con 0x7fd2880a3ed0 2024-04-25T21:08:39.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.417+0000 7fd2a49ad700 1 -- 172.21.15.37:0/4045553176 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258091 (secure 0 0 0) 0x7fd2a019c5d0 con 0x7fd2880a3ed0 2024-04-25T21:08:39.421 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 msgr2=0x7fd2880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 0x7fd2880a6390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd29000ac50 tx=0x7fd29000a380 comp rx=0 tx=0).stop 2024-04-25T21:08:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 msgr2=0x7fd2a019fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd298006070 tx=0x7fd298004e60 comp rx=0 tx=0).stop 2024-04-25T21:08:39.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 shutdown_connections 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2880a3ed0 0x7fd2880a6390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2a010f4f0 0x7fd2a019fb40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a006a750 0x7fd2a019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 --2- 172.21.15.37:0/4045553176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a006b070 0x7fd2a019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 >> 172.21.15.37:0/4045553176 conn(0x7fd2a00788f0 msgr2=0x7fd2a0110940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 shutdown_connections 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:39.421+0000 7fd2a59af700 1 -- 172.21.15.37:0/4045553176 wait complete. 2024-04-25T21:08:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T21:08:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:39 smithi037 ceph-mon[29195]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-25T21:08:39.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:39 smithi098 ceph-mon[32569]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-25T21:08:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:39 smithi177 ceph-mon[33043]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-25T21:08:40.171 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T21:08:38.115111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391575,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2574,"num_read_kb":1722,"num_write":718,"num_write_kb":5331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1042,"ondisk_log_size":1042,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365868,"kb_used_data":12516,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575956,"statfs":{"total":1151940427776,"available":1151565778944,"internally_reserved":0,"allocated":12816384,"data_stored":10033437,"data_compressed":245331,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199934,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":407,"num_read_kb":471,"num_write":153,"num_write_kb":921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.989582"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362557+0000","last_change":"2024-04-25T21:05:01.665417+0000","last_active":"2024-04-25T21:08:28.362557+0000","last_peered":"2024-04-25T21:08:28.362557+0000","last_clean":"2024-04-25T21:08:28.362557+0000","last_became_active":"2024-04-25T21:05:01.665131+0000","last_became_peered":"2024-04-25T21:05:01.665131+0000","last_unstale":"2024-04-25T21:08:28.362557+0000","last_undegraded":"2024-04-25T21:08:28.362557+0000","last_fullsized":"2024-04-25T21:08:28.362557+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:00:28.863716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362924+0000","last_change":"2024-04-25T21:05:01.667765+0000","last_active":"2024-04-25T21:08:28.362924+0000","last_peered":"2024-04-25T21:08:28.362924+0000","last_clean":"2024-04-25T21:08:28.362924+0000","last_became_active":"2024-04-25T21:05:01.665481+0000","last_became_peered":"2024-04-25T21:05:01.665481+0000","last_unstale":"2024-04-25T21:08:28.362924+0000","last_undegraded":"2024-04-25T21:08:28.362924+0000","last_fullsized":"2024-04-25T21:08:28.362924+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:07.044132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.665325+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.664955+0000","last_became_peered":"2024-04-25T21:05:01.664955+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:00:05.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363987+0000","last_change":"2024-04-25T21:05:01.663677+0000","last_active":"2024-04-25T21:08:28.363987+0000","last_peered":"2024-04-25T21:08:28.363987+0000","last_clean":"2024-04-25T21:08:28.363987+0000","last_became_active":"2024-04-25T21:05:01.662657+0000","last_became_peered":"2024-04-25T21:05:01.662657+0000","last_unstale":"2024-04-25T21:08:28.363987+0000","last_undegraded":"2024-04-25T21:08:28.363987+0000","last_fullsized":"2024-04-25T21:08:28.363987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:22:07.833995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195181+0000","last_change":"2024-04-25T21:05:01.653880+0000","last_active":"2024-04-25T21:08:28.195181+0000","last_peered":"2024-04-25T21:08:28.195181+0000","last_clean":"2024-04-25T21:08:28.195181+0000","last_became_active":"2024-04-25T21:05:01.653453+0000","last_became_peered":"2024-04-25T21:05:01.653453+0000","last_unstale":"2024-04-25T21:08:28.195181+0000","last_undegraded":"2024-04-25T21:08:28.195181+0000","last_fullsized":"2024-04-25T21:08:28.195181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:54:41.819981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205898+0000","last_change":"2024-04-25T21:05:01.658960+0000","last_active":"2024-04-25T21:08:28.205898+0000","last_peered":"2024-04-25T21:08:28.205898+0000","last_clean":"2024-04-25T21:08:28.205898+0000","last_became_active":"2024-04-25T21:05:01.658578+0000","last_became_peered":"2024-04-25T21:05:01.658578+0000","last_unstale":"2024-04-25T21:08:28.205898+0000","last_undegraded":"2024-04-25T21:08:28.205898+0000","last_fullsized":"2024-04-25T21:08:28.205898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:47.704474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357638+0000","last_change":"2024-04-25T21:05:01.650598+0000","last_active":"2024-04-25T21:08:28.357638+0000","last_peered":"2024-04-25T21:08:28.357638+0000","last_clean":"2024-04-25T21:08:28.357638+0000","last_became_active":"2024-04-25T21:05:01.650194+0000","last_became_peered":"2024-04-25T21:05:01.650194+0000","last_unstale":"2024-04-25T21:08:28.357638+0000","last_undegraded":"2024-04-25T21:08:28.357638+0000","last_fullsized":"2024-04-25T21:08:28.357638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:36.523174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194847+0000","last_change":"2024-04-25T21:05:01.651589+0000","last_active":"2024-04-25T21:08:28.194847+0000","last_peered":"2024-04-25T21:08:28.194847+0000","last_clean":"2024-04-25T21:08:28.194847+0000","last_became_active":"2024-04-25T21:05:01.651287+0000","last_became_peered":"2024-04-25T21:05:01.651287+0000","last_unstale":"2024-04-25T21:08:28.194847+0000","last_undegraded":"2024-04-25T21:08:28.194847+0000","last_fullsized":"2024-04-25T21:08:28.194847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:04:19.756147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101029+0000","last_change":"2024-04-25T21:05:01.656650+0000","last_active":"2024-04-25T21:08:28.101029+0000","last_peered":"2024-04-25T21:08:28.101029+0000","last_clean":"2024-04-25T21:08:28.101029+0000","last_became_active":"2024-04-25T21:05:01.655973+0000","last_became_peered":"2024-04-25T21:05:01.655973+0000","last_unstale":"2024-04-25T21:08:28.101029+0000","last_undegraded":"2024-04-25T21:08:28.101029+0000","last_fullsized":"2024-04-25T21:08:28.101029+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:33.128500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206257+0000","last_change":"2024-04-25T21:05:01.657218+0000","last_active":"2024-04-25T21:08:28.206257+0000","last_peered":"2024-04-25T21:08:28.206257+0000","last_clean":"2024-04-25T21:08:28.206257+0000","last_became_active":"2024-04-25T21:05:01.656873+0000","last_became_peered":"2024-04-25T21:05:01.656873+0000","last_unstale":"2024-04-25T21:08:28.206257+0000","last_undegraded":"2024-04-25T21:08:28.206257+0000","last_fullsized":"2024-04-25T21:08:28.206257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:23.441862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358006+0000","last_change":"2024-04-25T21:05:01.659502+0000","last_active":"2024-04-25T21:08:28.358006+0000","last_peered":"2024-04-25T21:08:28.358006+0000","last_clean":"2024-04-25T21:08:28.358006+0000","last_became_active":"2024-04-25T21:05:01.659161+0000","last_became_peered":"2024-04-25T21:05:01.659161+0000","last_unstale":"2024-04-25T21:08:28.358006+0000","last_undegraded":"2024-04-25T21:08:28.358006+0000","last_fullsized":"2024-04-25T21:08:28.358006+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:55:02.052460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363955+0000","last_change":"2024-04-25T21:05:01.664626+0000","last_active":"2024-04-25T21:08:28.363955+0000","last_peered":"2024-04-25T21:08:28.363955+0000","last_clean":"2024-04-25T21:08:28.363955+0000","last_became_active":"2024-04-25T21:05:01.663630+0000","last_became_peered":"2024-04-25T21:05:01.663630+0000","last_unstale":"2024-04-25T21:08:28.363955+0000","last_undegraded":"2024-04-25T21:08:28.363955+0000","last_fullsized":"2024-04-25T21:08:28.363955+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:16:04.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360589+0000","last_change":"2024-04-25T21:05:01.658951+0000","last_active":"2024-04-25T21:08:28.360589+0000","last_peered":"2024-04-25T21:08:28.360589+0000","last_clean":"2024-04-25T21:08:28.360589+0000","last_became_active":"2024-04-25T21:05:01.657581+0000","last_became_peered":"2024-04-25T21:05:01.657581+0000","last_unstale":"2024-04-25T21:08:28.360589+0000","last_undegraded":"2024-04-25T21:08:28.360589+0000","last_fullsized":"2024-04-25T21:08:28.360589+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:21:05.333594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362450+0000","last_change":"2024-04-25T21:05:01.663187+0000","last_active":"2024-04-25T21:08:28.362450+0000","last_peered":"2024-04-25T21:08:28.362450+0000","last_clean":"2024-04-25T21:08:28.362450+0000","last_became_active":"2024-04-25T21:05:01.662856+0000","last_became_peered":"2024-04-25T21:05:01.662856+0000","last_unstale":"2024-04-25T21:08:28.362450+0000","last_undegraded":"2024-04-25T21:08:28.362450+0000","last_fullsized":"2024-04-25T21:08:28.362450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:39.382076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362235+0000","last_change":"2024-04-25T21:05:35.818318+0000","last_active":"2024-04-25T21:08:28.362235+0000","last_peered":"2024-04-25T21:08:28.362235+0000","last_clean":"2024-04-25T21:08:28.362235+0000","last_became_active":"2024-04-25T21:05:35.817961+0000","last_became_peered":"2024-04-25T21:05:35.817961+0000","last_unstale":"2024-04-25T21:08:28.362235+0000","last_undegraded":"2024-04-25T21:08:28.362235+0000","last_fullsized":"2024-04-25T21:08:28.362235+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:51:33.197983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100450+0000","last_change":"2024-04-25T21:05:01.663381+0000","last_active":"2024-04-25T21:08:28.100450+0000","last_peered":"2024-04-25T21:08:28.100450+0000","last_clean":"2024-04-25T21:08:28.100450+0000","last_became_active":"2024-04-25T21:05:01.663059+0000","last_became_peered":"2024-04-25T21:05:01.663059+0000","last_unstale":"2024-04-25T21:08:28.100450+0000","last_undegraded":"2024-04-25T21:08:28.100450+0000","last_fullsized":"2024-04-25T21:08:28.100450+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:18:46.493422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362697+0000","last_change":"2024-04-25T21:05:01.666073+0000","last_active":"2024-04-25T21:08:28.362697+0000","last_peered":"2024-04-25T21:08:28.362697+0000","last_clean":"2024-04-25T21:08:28.362697+0000","last_became_active":"2024-04-25T21:05:01.665799+0000","last_became_peered":"2024-04-25T21:05:01.665799+0000","last_unstale":"2024-04-25T21:08:28.362697+0000","last_undegraded":"2024-04-25T21:08:28.362697+0000","last_fullsized":"2024-04-25T21:08:28.362697+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:40:16.169588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.099215+0000","last_change":"2024-04-25T21:05:01.653019+0000","last_active":"2024-04-25T21:08:28.099215+0000","last_peered":"2024-04-25T21:08:28.099215+0000","last_clean":"2024-04-25T21:08:28.099215+0000","last_became_active":"2024-04-25T21:05:01.652676+0000","last_became_peered":"2024-04-25T21:05:01.652676+0000","last_unstale":"2024-04-25T21:08:28.099215+0000","last_undegraded":"2024-04-25T21:08:28.099215+0000","last_fullsized":"2024-04-25T21:08:28.099215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:29:49.385644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101330+0000","last_change":"2024-04-25T21:05:01.654440+0000","last_active":"2024-04-25T21:08:28.101330+0000","last_peered":"2024-04-25T21:08:28.101330+0000","last_clean":"2024-04-25T21:08:28.101330+0000","last_became_active":"2024-04-25T21:05:01.654116+0000","last_became_peered":"2024-04-25T21:05:01.654116+0000","last_unstale":"2024-04-25T21:08:28.101330+0000","last_undegraded":"2024-04-25T21:08:28.101330+0000","last_fullsized":"2024-04-25T21:08:28.101330+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:39.627733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'14","reported_seq":238,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.511922+0000","last_change":"2024-04-25T21:04:57.620313+0000","last_active":"2024-04-25T21:08:32.511922+0000","last_peered":"2024-04-25T21:08:32.511922+0000","last_clean":"2024-04-25T21:08:32.511922+0000","last_became_active":"2024-04-25T21:04:57.619960+0000","last_became_peered":"2024-04-25T21:04:57.619960+0000","last_unstale":"2024-04-25T21:08:32.511922+0000","last_undegraded":"2024-04-25T21:08:32.511922+0000","last_fullsized":"2024-04-25T21:08:32.511922+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:42.002490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195064+0000","last_change":"2024-04-25T21:05:01.647577+0000","last_active":"2024-04-25T21:08:28.195064+0000","last_peered":"2024-04-25T21:08:28.195064+0000","last_clean":"2024-04-25T21:08:28.195064+0000","last_became_active":"2024-04-25T21:05:01.647219+0000","last_became_peered":"2024-04-25T21:05:01.647219+0000","last_unstale":"2024-04-25T21:08:28.195064+0000","last_undegraded":"2024-04-25T21:08:28.195064+0000","last_fullsized":"2024-04-25T21:08:28.195064+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:28.676266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195494+0000","last_change":"2024-04-25T21:04:57.844645+0000","last_active":"2024-04-25T21:08:28.195494+0000","last_peered":"2024-04-25T21:08:28.195494+0000","last_clean":"2024-04-25T21:08:28.195494+0000","last_became_active":"2024-04-25T21:04:57.842892+0000","last_became_peered":"2024-04-25T21:04:57.842892+0000","last_unstale":"2024-04-25T21:08:28.195494+0000","last_undegraded":"2024-04-25T21:08:28.195494+0000","last_fullsized":"2024-04-25T21:08:28.195494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:50.029918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363253+0000","last_change":"2024-04-25T21:05:01.663182+0000","last_active":"2024-04-25T21:08:28.363253+0000","last_peered":"2024-04-25T21:08:28.363253+0000","last_clean":"2024-04-25T21:08:28.363253+0000","last_became_active":"2024-04-25T21:05:01.662835+0000","last_became_peered":"2024-04-25T21:05:01.662835+0000","last_unstale":"2024-04-25T21:08:28.363253+0000","last_undegraded":"2024-04-25T21:08:28.363253+0000","last_fullsized":"2024-04-25T21:08:28.363253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:03.632510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195601+0000","last_change":"2024-04-25T21:04:57.624851+0000","last_active":"2024-04-25T21:08:28.195601+0000","last_peered":"2024-04-25T21:08:28.195601+0000","last_clean":"2024-04-25T21:08:28.195601+0000","last_became_active":"2024-04-25T21:04:57.624483+0000","last_became_peered":"2024-04-25T21:04:57.624483+0000","last_unstale":"2024-04-25T21:08:28.195601+0000","last_undegraded":"2024-04-25T21:08:28.195601+0000","last_fullsized":"2024-04-25T21:08:28.195601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:27.015368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100566+0000","last_change":"2024-04-25T21:05:01.658486+0000","last_active":"2024-04-25T21:08:28.100566+0000","last_peered":"2024-04-25T21:08:28.100566+0000","last_clean":"2024-04-25T21:08:28.100566+0000","last_became_active":"2024-04-25T21:05:01.658079+0000","last_became_peered":"2024-04-25T21:05:01.658079+0000","last_unstale":"2024-04-25T21:08:28.100566+0000","last_undegraded":"2024-04-25T21:08:28.100566+0000","last_fullsized":"2024-04-25T21:08:28.100566+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:03:28.567722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362183+0000","last_change":"2024-04-25T21:05:35.823283+0000","last_active":"2024-04-25T21:08:28.362183+0000","last_peered":"2024-04-25T21:08:28.362183+0000","last_clean":"2024-04-25T21:08:28.362183+0000","last_became_active":"2024-04-25T21:05:35.816821+0000","last_became_peered":"2024-04-25T21:05:35.816821+0000","last_unstale":"2024-04-25T21:08:28.362183+0000","last_undegraded":"2024-04-25T21:08:28.362183+0000","last_fullsized":"2024-04-25T21:08:28.362183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:58.481386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203110+0000","last_change":"2024-04-25T21:05:01.656987+0000","last_active":"2024-04-25T21:08:28.203110+0000","last_peered":"2024-04-25T21:08:28.203110+0000","last_clean":"2024-04-25T21:08:28.203110+0000","last_became_active":"2024-04-25T21:05:01.656663+0000","last_became_peered":"2024-04-25T21:05:01.656663+0000","last_unstale":"2024-04-25T21:08:28.203110+0000","last_undegraded":"2024-04-25T21:08:28.203110+0000","last_fullsized":"2024-04-25T21:08:28.203110+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:58.164243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'8","reported_seq":236,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.527332+0000","last_change":"2024-04-25T21:04:57.621818+0000","last_active":"2024-04-25T21:08:32.527332+0000","last_peered":"2024-04-25T21:08:32.527332+0000","last_clean":"2024-04-25T21:08:32.527332+0000","last_became_active":"2024-04-25T21:04:57.621397+0000","last_became_peered":"2024-04-25T21:04:57.621397+0000","last_unstale":"2024-04-25T21:08:32.527332+0000","last_undegraded":"2024-04-25T21:08:32.527332+0000","last_fullsized":"2024-04-25T21:08:32.527332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:16:51.187755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":199,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195157+0000","last_change":"2024-04-25T21:05:01.661648+0000","last_active":"2024-04-25T21:08:28.195157+0000","last_peered":"2024-04-25T21:08:28.195157+0000","last_clean":"2024-04-25T21:08:28.195157+0000","last_became_active":"2024-04-25T21:05:01.661315+0000","last_became_peered":"2024-04-25T21:05:01.661315+0000","last_unstale":"2024-04-25T21:08:28.195157+0000","last_undegraded":"2024-04-25T21:08:28.195157+0000","last_fullsized":"2024-04-25T21:08:28.195157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:51.361398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205818+0000","last_change":"2024-04-25T21:04:57.621208+0000","last_active":"2024-04-25T21:08:28.205818+0000","last_peered":"2024-04-25T21:08:28.205818+0000","last_clean":"2024-04-25T21:08:28.205818+0000","last_became_active":"2024-04-25T21:04:57.620895+0000","last_became_peered":"2024-04-25T21:04:57.620895+0000","last_unstale":"2024-04-25T21:08:28.205818+0000","last_undegraded":"2024-04-25T21:08:28.205818+0000","last_fullsized":"2024-04-25T21:08:28.205818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:12.991724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203072+0000","last_change":"2024-04-25T21:05:01.658424+0000","last_active":"2024-04-25T21:08:28.203072+0000","last_peered":"2024-04-25T21:08:28.203072+0000","last_clean":"2024-04-25T21:08:28.203072+0000","last_became_active":"2024-04-25T21:05:01.657673+0000","last_became_peered":"2024-04-25T21:05:01.657673+0000","last_unstale":"2024-04-25T21:08:28.203072+0000","last_undegraded":"2024-04-25T21:08:28.203072+0000","last_fullsized":"2024-04-25T21:08:28.203072+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:12.400280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359792+0000","last_change":"2024-04-25T21:04:57.617496+0000","last_active":"2024-04-25T21:08:28.359792+0000","last_peered":"2024-04-25T21:08:28.359792+0000","last_clean":"2024-04-25T21:08:28.359792+0000","last_became_active":"2024-04-25T21:04:57.617171+0000","last_became_peered":"2024-04-25T21:04:57.617171+0000","last_unstale":"2024-04-25T21:08:28.359792+0000","last_undegraded":"2024-04-25T21:08:28.359792+0000","last_fullsized":"2024-04-25T21:08:28.359792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:09:52.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194771+0000","last_change":"2024-04-25T21:05:01.647124+0000","last_active":"2024-04-25T21:08:28.194771+0000","last_peered":"2024-04-25T21:08:28.194771+0000","last_clean":"2024-04-25T21:08:28.194771+0000","last_became_active":"2024-04-25T21:05:01.646790+0000","last_became_peered":"2024-04-25T21:05:01.646790+0000","last_unstale":"2024-04-25T21:08:28.194771+0000","last_undegraded":"2024-04-25T21:08:28.194771+0000","last_fullsized":"2024-04-25T21:08:28.194771+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:46.208022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202732+0000","last_change":"2024-04-25T21:04:57.620912+0000","last_active":"2024-04-25T21:08:28.202732+0000","last_peered":"2024-04-25T21:08:28.202732+0000","last_clean":"2024-04-25T21:08:28.202732+0000","last_became_active":"2024-04-25T21:04:57.620466+0000","last_became_peered":"2024-04-25T21:04:57.620466+0000","last_unstale":"2024-04-25T21:08:28.202732+0000","last_undegraded":"2024-04-25T21:08:28.202732+0000","last_fullsized":"2024-04-25T21:08:28.202732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:05.045563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098804+0000","last_change":"2024-04-25T21:05:01.652610+0000","last_active":"2024-04-25T21:08:28.098804+0000","last_peered":"2024-04-25T21:08:28.098804+0000","last_clean":"2024-04-25T21:08:28.098804+0000","last_became_active":"2024-04-25T21:05:01.652257+0000","last_became_peered":"2024-04-25T21:05:01.652257+0000","last_unstale":"2024-04-25T21:08:28.098804+0000","last_undegraded":"2024-04-25T21:08:28.098804+0000","last_fullsized":"2024-04-25T21:08:28.098804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:23.072739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":425,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:33.528301+0000","last_change":"2024-04-25T21:04:57.623241+0000","last_active":"2024-04-25T21:08:33.528301+0000","last_peered":"2024-04-25T21:08:33.528301+0000","last_clean":"2024-04-25T21:08:33.528301+0000","last_became_active":"2024-04-25T21:04:57.622852+0000","last_became_peered":"2024-04-25T21:04:57.622852+0000","last_unstale":"2024-04-25T21:08:33.528301+0000","last_undegraded":"2024-04-25T21:08:33.528301+0000","last_fullsized":"2024-04-25T21:08:33.528301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:34:53.614555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":388,"num_read_kb":388,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101031+0000","last_change":"2024-04-25T21:05:01.658913+0000","last_active":"2024-04-25T21:08:28.101031+0000","last_peered":"2024-04-25T21:08:28.101031+0000","last_clean":"2024-04-25T21:08:28.101031+0000","last_became_active":"2024-04-25T21:05:01.658585+0000","last_became_peered":"2024-04-25T21:05:01.658585+0000","last_unstale":"2024-04-25T21:08:28.101031+0000","last_undegraded":"2024-04-25T21:08:28.101031+0000","last_fullsized":"2024-04-25T21:08:28.101031+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:59:51.391007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358082+0000","last_change":"2024-04-25T21:04:57.621524+0000","last_active":"2024-04-25T21:08:28.358082+0000","last_peered":"2024-04-25T21:08:28.358082+0000","last_clean":"2024-04-25T21:08:28.358082+0000","last_became_active":"2024-04-25T21:04:57.621103+0000","last_became_peered":"2024-04-25T21:04:57.621103+0000","last_unstale":"2024-04-25T21:08:28.358082+0000","last_undegraded":"2024-04-25T21:08:28.358082+0000","last_fullsized":"2024-04-25T21:08:28.358082+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:15:38.999492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363081+0000","last_change":"2024-04-25T21:05:01.662447+0000","last_active":"2024-04-25T21:08:28.363081+0000","last_peered":"2024-04-25T21:08:28.363081+0000","last_clean":"2024-04-25T21:08:28.363081+0000","last_became_active":"2024-04-25T21:05:01.662038+0000","last_became_peered":"2024-04-25T21:05:01.662038+0000","last_unstale":"2024-04-25T21:08:28.363081+0000","last_undegraded":"2024-04-25T21:08:28.363081+0000","last_fullsized":"2024-04-25T21:08:28.363081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:37:19.444088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.203162+0000","last_change":"2024-04-25T21:04:57.844854+0000","last_active":"2024-04-25T21:08:28.203162+0000","last_peered":"2024-04-25T21:08:28.203162+0000","last_clean":"2024-04-25T21:08:28.203162+0000","last_became_active":"2024-04-25T21:04:57.844180+0000","last_became_peered":"2024-04-25T21:04:57.844180+0000","last_unstale":"2024-04-25T21:08:28.203162+0000","last_undegraded":"2024-04-25T21:08:28.203162+0000","last_fullsized":"2024-04-25T21:08:28.203162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:12.668800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100668+0000","last_change":"2024-04-25T21:05:01.660709+0000","last_active":"2024-04-25T21:08:28.100668+0000","last_peered":"2024-04-25T21:08:28.100668+0000","last_clean":"2024-04-25T21:08:28.100668+0000","last_became_active":"2024-04-25T21:05:01.660072+0000","last_became_peered":"2024-04-25T21:05:01.660072+0000","last_unstale":"2024-04-25T21:08:28.100668+0000","last_undegraded":"2024-04-25T21:08:28.100668+0000","last_fullsized":"2024-04-25T21:08:28.100668+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:27:04.644121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'41","reported_seq":471,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:33.546339+0000","last_change":"2024-04-25T21:04:57.622765+0000","last_active":"2024-04-25T21:08:33.546339+0000","last_peered":"2024-04-25T21:08:33.546339+0000","last_clean":"2024-04-25T21:08:33.546339+0000","last_became_active":"2024-04-25T21:04:57.622417+0000","last_became_peered":"2024-04-25T21:04:57.622417+0000","last_unstale":"2024-04-25T21:08:33.546339+0000","last_undegraded":"2024-04-25T21:08:33.546339+0000","last_fullsized":"2024-04-25T21:08:33.546339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:03:03.866048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":401,"num_read_kb":268,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195284+0000","last_change":"2024-04-25T21:05:01.657800+0000","last_active":"2024-04-25T21:08:28.195284+0000","last_peered":"2024-04-25T21:08:28.195284+0000","last_clean":"2024-04-25T21:08:28.195284+0000","last_became_active":"2024-04-25T21:05:01.657510+0000","last_became_peered":"2024-04-25T21:05:01.657510+0000","last_unstale":"2024-04-25T21:08:28.195284+0000","last_undegraded":"2024-04-25T21:08:28.195284+0000","last_fullsized":"2024-04-25T21:08:28.195284+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.445283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101066+0000","last_change":"2024-04-25T21:04:57.619437+0000","last_active":"2024-04-25T21:08:28.101066+0000","last_peered":"2024-04-25T21:08:28.101066+0000","last_clean":"2024-04-25T21:08:28.101066+0000","last_became_active":"2024-04-25T21:04:57.619129+0000","last_became_peered":"2024-04-25T21:04:57.619129+0000","last_unstale":"2024-04-25T21:08:28.101066+0000","last_undegraded":"2024-04-25T21:08:28.101066+0000","last_fullsized":"2024-04-25T21:08:28.101066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:07.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360236+0000","last_change":"2024-04-25T21:05:01.656519+0000","last_active":"2024-04-25T21:08:28.360236+0000","last_peered":"2024-04-25T21:08:28.360236+0000","last_clean":"2024-04-25T21:08:28.360236+0000","last_became_active":"2024-04-25T21:05:01.656214+0000","last_became_peered":"2024-04-25T21:05:01.656214+0000","last_unstale":"2024-04-25T21:08:28.360236+0000","last_undegraded":"2024-04-25T21:08:28.360236+0000","last_fullsized":"2024-04-25T21:08:28.360236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:45.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202903+0000","last_change":"2024-04-25T21:04:57.621346+0000","last_active":"2024-04-25T21:08:28.202903+0000","last_peered":"2024-04-25T21:08:28.202903+0000","last_clean":"2024-04-25T21:08:28.202903+0000","last_became_active":"2024-04-25T21:04:57.620494+0000","last_became_peered":"2024-04-25T21:04:57.620494+0000","last_unstale":"2024-04-25T21:08:28.202903+0000","last_undegraded":"2024-04-25T21:08:28.202903+0000","last_fullsized":"2024-04-25T21:08:28.202903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:06.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357857+0000","last_change":"2024-04-25T21:05:35.848492+0000","last_active":"2024-04-25T21:08:28.357857+0000","last_peered":"2024-04-25T21:08:28.357857+0000","last_clean":"2024-04-25T21:08:28.357857+0000","last_became_active":"2024-04-25T21:05:35.848099+0000","last_became_peered":"2024-04-25T21:05:35.848099+0000","last_unstale":"2024-04-25T21:08:28.357857+0000","last_undegraded":"2024-04-25T21:08:28.357857+0000","last_fullsized":"2024-04-25T21:08:28.357857+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:52:34.449963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":234,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512998+0000","last_change":"2024-04-25T21:04:57.620267+0000","last_active":"2024-04-25T21:08:32.512998+0000","last_peered":"2024-04-25T21:08:32.512998+0000","last_clean":"2024-04-25T21:08:32.512998+0000","last_became_active":"2024-04-25T21:04:57.619898+0000","last_became_peered":"2024-04-25T21:04:57.619898+0000","last_unstale":"2024-04-25T21:08:32.512998+0000","last_undegraded":"2024-04-25T21:08:32.512998+0000","last_fullsized":"2024-04-25T21:08:32.512998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357601+0000","last_change":"2024-04-25T21:05:01.653148+0000","last_active":"2024-04-25T21:08:28.357601+0000","last_peered":"2024-04-25T21:08:28.357601+0000","last_clean":"2024-04-25T21:08:28.357601+0000","last_became_active":"2024-04-25T21:05:01.652821+0000","last_became_peered":"2024-04-25T21:05:01.652821+0000","last_unstale":"2024-04-25T21:08:28.357601+0000","last_undegraded":"2024-04-25T21:08:28.357601+0000","last_fullsized":"2024-04-25T21:08:28.357601+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:56:25.351973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195075+0000","last_change":"2024-04-25T21:04:57.845442+0000","last_active":"2024-04-25T21:08:28.195075+0000","last_peered":"2024-04-25T21:08:28.195075+0000","last_clean":"2024-04-25T21:08:28.195075+0000","last_became_active":"2024-04-25T21:04:57.844741+0000","last_became_peered":"2024-04-25T21:04:57.844741+0000","last_unstale":"2024-04-25T21:08:28.195075+0000","last_undegraded":"2024-04-25T21:08:28.195075+0000","last_fullsized":"2024-04-25T21:08:28.195075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:02.518798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202900+0000","last_change":"2024-04-25T21:05:01.654587+0000","last_active":"2024-04-25T21:08:28.202900+0000","last_peered":"2024-04-25T21:08:28.202900+0000","last_clean":"2024-04-25T21:08:28.202900+0000","last_became_active":"2024-04-25T21:05:01.654236+0000","last_became_peered":"2024-04-25T21:05:01.654236+0000","last_unstale":"2024-04-25T21:08:28.202900+0000","last_undegraded":"2024-04-25T21:08:28.202900+0000","last_fullsized":"2024-04-25T21:08:28.202900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:14.504202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'3","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195524+0000","last_change":"2024-04-25T21:04:57.622809+0000","last_active":"2024-04-25T21:08:28.195524+0000","last_peered":"2024-04-25T21:08:28.195524+0000","last_clean":"2024-04-25T21:08:28.195524+0000","last_became_active":"2024-04-25T21:04:57.622461+0000","last_became_peered":"2024-04-25T21:04:57.622461+0000","last_unstale":"2024-04-25T21:08:28.195524+0000","last_undegraded":"2024-04-25T21:08:28.195524+0000","last_fullsized":"2024-04-25T21:08:28.195524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:45.419917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362402+0000","last_change":"2024-04-25T21:05:01.656151+0000","last_active":"2024-04-25T21:08:28.362402+0000","last_peered":"2024-04-25T21:08:28.362402+0000","last_clean":"2024-04-25T21:08:28.362402+0000","last_became_active":"2024-04-25T21:05:01.655782+0000","last_became_peered":"2024-04-25T21:05:01.655782+0000","last_unstale":"2024-04-25T21:08:28.362402+0000","last_undegraded":"2024-04-25T21:08:28.362402+0000","last_fullsized":"2024-04-25T21:08:28.362402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:16.464283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.098985+0000","last_change":"2024-04-25T21:04:57.843395+0000","last_active":"2024-04-25T21:08:28.098985+0000","last_peered":"2024-04-25T21:08:28.098985+0000","last_clean":"2024-04-25T21:08:28.098985+0000","last_became_active":"2024-04-25T21:04:57.842870+0000","last_became_peered":"2024-04-25T21:04:57.842870+0000","last_unstale":"2024-04-25T21:08:28.098985+0000","last_undegraded":"2024-04-25T21:08:28.098985+0000","last_fullsized":"2024-04-25T21:08:28.098985+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:14.710455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.206100+0000","last_change":"2024-04-25T21:05:01.658152+0000","last_active":"2024-04-25T21:08:28.206100+0000","last_peered":"2024-04-25T21:08:28.206100+0000","last_clean":"2024-04-25T21:08:28.206100+0000","last_became_active":"2024-04-25T21:05:01.657703+0000","last_became_peered":"2024-04-25T21:05:01.657703+0000","last_unstale":"2024-04-25T21:08:28.206100+0000","last_undegraded":"2024-04-25T21:08:28.206100+0000","last_fullsized":"2024-04-25T21:08:28.206100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:34:07.325336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'6","reported_seq":230,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512665+0000","last_change":"2024-04-25T21:04:57.624169+0000","last_active":"2024-04-25T21:08:32.512665+0000","last_peered":"2024-04-25T21:08:32.512665+0000","last_clean":"2024-04-25T21:08:32.512665+0000","last_became_active":"2024-04-25T21:04:57.620364+0000","last_became_peered":"2024-04-25T21:04:57.620364+0000","last_unstale":"2024-04-25T21:08:32.512665+0000","last_undegraded":"2024-04-25T21:08:32.512665+0000","last_fullsized":"2024-04-25T21:08:32.512665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:31.620309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.205831+0000","last_change":"2024-04-25T21:05:01.662003+0000","last_active":"2024-04-25T21:08:28.205831+0000","last_peered":"2024-04-25T21:08:28.205831+0000","last_clean":"2024-04-25T21:08:28.205831+0000","last_became_active":"2024-04-25T21:05:01.661636+0000","last_became_peered":"2024-04-25T21:05:01.661636+0000","last_unstale":"2024-04-25T21:08:28.205831+0000","last_undegraded":"2024-04-25T21:08:28.205831+0000","last_fullsized":"2024-04-25T21:08:28.205831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:03.067766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202984+0000","last_change":"2024-04-25T21:05:01.658619+0000","last_active":"2024-04-25T21:08:28.202984+0000","last_peered":"2024-04-25T21:08:28.202984+0000","last_clean":"2024-04-25T21:08:28.202984+0000","last_became_active":"2024-04-25T21:05:01.658299+0000","last_became_peered":"2024-04-25T21:05:01.658299+0000","last_unstale":"2024-04-25T21:08:28.202984+0000","last_undegraded":"2024-04-25T21:08:28.202984+0000","last_fullsized":"2024-04-25T21:08:28.202984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:33.622931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.512521+0000","last_change":"2024-04-25T21:04:57.843419+0000","last_active":"2024-04-25T21:08:32.512521+0000","last_peered":"2024-04-25T21:08:32.512521+0000","last_clean":"2024-04-25T21:08:32.512521+0000","last_became_active":"2024-04-25T21:04:57.843001+0000","last_became_peered":"2024-04-25T21:04:57.843001+0000","last_unstale":"2024-04-25T21:08:32.512521+0000","last_undegraded":"2024-04-25T21:08:32.512521+0000","last_fullsized":"2024-04-25T21:08:32.512521+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.997252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":195,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194859+0000","last_change":"2024-04-25T21:05:01.642728+0000","last_active":"2024-04-25T21:08:28.194859+0000","last_peered":"2024-04-25T21:08:28.194859+0000","last_clean":"2024-04-25T21:08:28.194859+0000","last_became_active":"2024-04-25T21:05:01.641883+0000","last_became_peered":"2024-04-25T21:05:01.641883+0000","last_unstale":"2024-04-25T21:08:28.194859+0000","last_undegraded":"2024-04-25T21:08:28.194859+0000","last_fullsized":"2024-04-25T21:08:28.194859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:17.975461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.363893+0000","last_change":"2024-04-25T21:04:57.619053+0000","last_active":"2024-04-25T21:08:28.363893+0000","last_peered":"2024-04-25T21:08:28.363893+0000","last_clean":"2024-04-25T21:08:28.363893+0000","last_became_active":"2024-04-25T21:04:57.618319+0000","last_became_peered":"2024-04-25T21:04:57.618319+0000","last_unstale":"2024-04-25T21:08:28.363893+0000","last_undegraded":"2024-04-25T21:08:28.363893+0000","last_fullsized":"2024-04-25T21:08:28.363893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:27.657175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'148","reported_seq":403,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.208758+0000","last_change":"2024-04-25T21:03:28.237472+0000","last_active":"2024-04-25T21:08:30.208758+0000","last_peered":"2024-04-25T21:08:30.208758+0000","last_clean":"2024-04-25T21:08:30.208758+0000","last_became_active":"2024-04-25T21:03:28.236987+0000","last_became_peered":"2024-04-25T21:03:28.236987+0000","last_unstale":"2024-04-25T21:08:30.208758+0000","last_undegraded":"2024-04-25T21:08:30.208758+0000","last_fullsized":"2024-04-25T21:08:30.208758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:00:00.551715+0000","last_clean_scrub_stamp":"2024-04-25T21:00:00.551715+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:20:51.136798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.359561+0000","last_change":"2024-04-25T21:05:01.655785+0000","last_active":"2024-04-25T21:08:28.359561+0000","last_peered":"2024-04-25T21:08:28.359561+0000","last_clean":"2024-04-25T21:08:28.359561+0000","last_became_active":"2024-04-25T21:05:01.655478+0000","last_became_peered":"2024-04-25T21:05:01.655478+0000","last_unstale":"2024-04-25T21:08:28.359561+0000","last_undegraded":"2024-04-25T21:08:28.359561+0000","last_fullsized":"2024-04-25T21:08:28.359561+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:57.685133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357841+0000","last_change":"2024-04-25T21:04:57.621596+0000","last_active":"2024-04-25T21:08:28.357841+0000","last_peered":"2024-04-25T21:08:28.357841+0000","last_clean":"2024-04-25T21:08:28.357841+0000","last_became_active":"2024-04-25T21:04:57.620903+0000","last_became_peered":"2024-04-25T21:04:57.620903+0000","last_unstale":"2024-04-25T21:08:28.357841+0000","last_undegraded":"2024-04-25T21:08:28.357841+0000","last_fullsized":"2024-04-25T21:08:28.357841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:59.035108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195361+0000","last_change":"2024-04-25T21:04:57.843571+0000","last_active":"2024-04-25T21:08:28.195361+0000","last_peered":"2024-04-25T21:08:28.195361+0000","last_clean":"2024-04-25T21:08:28.195361+0000","last_became_active":"2024-04-25T21:04:57.843201+0000","last_became_peered":"2024-04-25T21:04:57.843201+0000","last_unstale":"2024-04-25T21:08:28.195361+0000","last_undegraded":"2024-04-25T21:08:28.195361+0000","last_fullsized":"2024-04-25T21:08:28.195361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:55:47.417383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360741+0000","last_change":"2024-04-25T21:05:01.653052+0000","last_active":"2024-04-25T21:08:28.360741+0000","last_peered":"2024-04-25T21:08:28.360741+0000","last_clean":"2024-04-25T21:08:28.360741+0000","last_became_active":"2024-04-25T21:05:01.652744+0000","last_became_peered":"2024-04-25T21:05:01.652744+0000","last_unstale":"2024-04-25T21:08:28.360741+0000","last_undegraded":"2024-04-25T21:08:28.360741+0000","last_fullsized":"2024-04-25T21:08:28.360741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:54.713978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":226,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.727113+0000","last_change":"2024-04-25T21:04:57.617773+0000","last_active":"2024-04-25T21:08:30.727113+0000","last_peered":"2024-04-25T21:08:30.727113+0000","last_clean":"2024-04-25T21:08:30.727113+0000","last_became_active":"2024-04-25T21:04:57.617328+0000","last_became_peered":"2024-04-25T21:04:57.617328+0000","last_unstale":"2024-04-25T21:08:30.727113+0000","last_undegraded":"2024-04-25T21:08:30.727113+0000","last_fullsized":"2024-04-25T21:08:30.727113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:44.148499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195491+0000","last_change":"2024-04-25T21:05:01.647866+0000","last_active":"2024-04-25T21:08:28.195491+0000","last_peered":"2024-04-25T21:08:28.195491+0000","last_clean":"2024-04-25T21:08:28.195491+0000","last_became_active":"2024-04-25T21:05:01.647163+0000","last_became_peered":"2024-04-25T21:05:01.647163+0000","last_unstale":"2024-04-25T21:08:28.195491+0000","last_undegraded":"2024-04-25T21:08:28.195491+0000","last_fullsized":"2024-04-25T21:08:28.195491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:22:35.572545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.364109+0000","last_change":"2024-04-25T21:04:58.289011+0000","last_active":"2024-04-25T21:08:28.364109+0000","last_peered":"2024-04-25T21:08:28.364109+0000","last_clean":"2024-04-25T21:08:28.364109+0000","last_became_active":"2024-04-25T21:04:58.288630+0000","last_became_peered":"2024-04-25T21:04:58.288630+0000","last_unstale":"2024-04-25T21:08:28.364109+0000","last_undegraded":"2024-04-25T21:08:28.364109+0000","last_fullsized":"2024-04-25T21:08:28.364109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:18.656830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362959+0000","last_change":"2024-04-25T21:05:01.668100+0000","last_active":"2024-04-25T21:08:28.362959+0000","last_peered":"2024-04-25T21:08:28.362959+0000","last_clean":"2024-04-25T21:08:28.362959+0000","last_became_active":"2024-04-25T21:05:01.667713+0000","last_became_peered":"2024-04-25T21:05:01.667713+0000","last_unstale":"2024-04-25T21:08:28.362959+0000","last_undegraded":"2024-04-25T21:08:28.362959+0000","last_fullsized":"2024-04-25T21:08:28.362959+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:01.365011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100466+0000","last_change":"2024-04-25T21:05:35.820411+0000","last_active":"2024-04-25T21:08:28.100466+0000","last_peered":"2024-04-25T21:08:28.100466+0000","last_clean":"2024-04-25T21:08:28.100466+0000","last_became_active":"2024-04-25T21:05:35.817447+0000","last_became_peered":"2024-04-25T21:05:35.817447+0000","last_unstale":"2024-04-25T21:08:28.100466+0000","last_undegraded":"2024-04-25T21:08:28.100466+0000","last_fullsized":"2024-04-25T21:08:28.100466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:07.590689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195078+0000","last_change":"2024-04-25T21:05:01.657668+0000","last_active":"2024-04-25T21:08:28.195078+0000","last_peered":"2024-04-25T21:08:28.195078+0000","last_clean":"2024-04-25T21:08:28.195078+0000","last_became_active":"2024-04-25T21:05:01.657342+0000","last_became_peered":"2024-04-25T21:05:01.657342+0000","last_unstale":"2024-04-25T21:08:28.195078+0000","last_undegraded":"2024-04-25T21:08:28.195078+0000","last_fullsized":"2024-04-25T21:08:28.195078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:39.842014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194864+0000","last_change":"2024-04-25T21:04:57.618505+0000","last_active":"2024-04-25T21:08:28.194864+0000","last_peered":"2024-04-25T21:08:28.194864+0000","last_clean":"2024-04-25T21:08:28.194864+0000","last_became_active":"2024-04-25T21:04:57.618170+0000","last_became_peered":"2024-04-25T21:04:57.618170+0000","last_unstale":"2024-04-25T21:08:28.194864+0000","last_undegraded":"2024-04-25T21:08:28.194864+0000","last_fullsized":"2024-04-25T21:08:28.194864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:52:09.976055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362215+0000","last_change":"2024-04-25T21:05:01.660743+0000","last_active":"2024-04-25T21:08:28.362215+0000","last_peered":"2024-04-25T21:08:28.362215+0000","last_clean":"2024-04-25T21:08:28.362215+0000","last_became_active":"2024-04-25T21:05:01.660396+0000","last_became_peered":"2024-04-25T21:05:01.660396+0000","last_unstale":"2024-04-25T21:08:28.362215+0000","last_undegraded":"2024-04-25T21:08:28.362215+0000","last_fullsized":"2024-04-25T21:08:28.362215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:21.554930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.769735+0000","last_change":"2024-04-25T21:04:57.618734+0000","last_active":"2024-04-25T21:08:28.769735+0000","last_peered":"2024-04-25T21:08:28.769735+0000","last_clean":"2024-04-25T21:08:28.769735+0000","last_became_active":"2024-04-25T21:04:57.617769+0000","last_became_peered":"2024-04-25T21:04:57.617769+0000","last_unstale":"2024-04-25T21:08:28.769735+0000","last_undegraded":"2024-04-25T21:08:28.769735+0000","last_fullsized":"2024-04-25T21:08:28.769735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:30:13.904505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195175+0000","last_change":"2024-04-25T21:05:01.666339+0000","last_active":"2024-04-25T21:08:28.195175+0000","last_peered":"2024-04-25T21:08:28.195175+0000","last_clean":"2024-04-25T21:08:28.195175+0000","last_became_active":"2024-04-25T21:05:01.666064+0000","last_became_peered":"2024-04-25T21:05:01.666064+0000","last_unstale":"2024-04-25T21:08:28.195175+0000","last_undegraded":"2024-04-25T21:08:28.195175+0000","last_fullsized":"2024-04-25T21:08:28.195175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:31.604076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195252+0000","last_change":"2024-04-25T21:04:57.625388+0000","last_active":"2024-04-25T21:08:28.195252+0000","last_peered":"2024-04-25T21:08:28.195252+0000","last_clean":"2024-04-25T21:08:28.195252+0000","last_became_active":"2024-04-25T21:04:57.624987+0000","last_became_peered":"2024-04-25T21:04:57.624987+0000","last_unstale":"2024-04-25T21:08:28.195252+0000","last_undegraded":"2024-04-25T21:08:28.195252+0000","last_fullsized":"2024-04-25T21:08:28.195252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:00.828041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357809+0000","last_change":"2024-04-25T21:05:01.657877+0000","last_active":"2024-04-25T21:08:28.357809+0000","last_peered":"2024-04-25T21:08:28.357809+0000","last_clean":"2024-04-25T21:08:28.357809+0000","last_became_active":"2024-04-25T21:05:01.657534+0000","last_became_peered":"2024-04-25T21:05:01.657534+0000","last_unstale":"2024-04-25T21:08:28.357809+0000","last_undegraded":"2024-04-25T21:08:28.357809+0000","last_fullsized":"2024-04-25T21:08:28.357809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:49:45.390444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'10","reported_seq":239,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:32.545655+0000","last_change":"2024-04-25T21:04:58.288972+0000","last_active":"2024-04-25T21:08:32.545655+0000","last_peered":"2024-04-25T21:08:32.545655+0000","last_clean":"2024-04-25T21:08:32.545655+0000","last_became_active":"2024-04-25T21:04:58.288396+0000","last_became_peered":"2024-04-25T21:04:58.288396+0000","last_unstale":"2024-04-25T21:08:32.545655+0000","last_undegraded":"2024-04-25T21:08:32.545655+0000","last_fullsized":"2024-04-25T21:08:32.545655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:40:54.951169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2792,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":6,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358171+0000","last_change":"2024-04-25T21:05:01.657921+0000","last_active":"2024-04-25T21:08:28.358171+0000","last_peered":"2024-04-25T21:08:28.358171+0000","last_clean":"2024-04-25T21:08:28.358171+0000","last_became_active":"2024-04-25T21:05:01.657562+0000","last_became_peered":"2024-04-25T21:05:01.657562+0000","last_unstale":"2024-04-25T21:08:28.358171+0000","last_undegraded":"2024-04-25T21:08:28.358171+0000","last_fullsized":"2024-04-25T21:08:28.358171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:18.900775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":127,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:30.369314+0000","last_change":"2024-04-25T21:04:57.619170+0000","last_active":"2024-04-25T21:08:30.369314+0000","last_peered":"2024-04-25T21:08:30.369314+0000","last_clean":"2024-04-25T21:08:30.369314+0000","last_became_active":"2024-04-25T21:04:57.618089+0000","last_became_peered":"2024-04-25T21:04:57.618089+0000","last_unstale":"2024-04-25T21:08:30.369314+0000","last_undegraded":"2024-04-25T21:08:30.369314+0000","last_fullsized":"2024-04-25T21:08:30.369314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:39:12.959062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.357893+0000","last_change":"2024-04-25T21:05:01.652423+0000","last_active":"2024-04-25T21:08:28.357893+0000","last_peered":"2024-04-25T21:08:28.357893+0000","last_clean":"2024-04-25T21:08:28.357893+0000","last_became_active":"2024-04-25T21:05:01.652086+0000","last_became_peered":"2024-04-25T21:05:01.652086+0000","last_unstale":"2024-04-25T21:08:28.357893+0000","last_undegraded":"2024-04-25T21:08:28.357893+0000","last_fullsized":"2024-04-25T21:08:28.357893+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:25.138928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360443+0000","last_change":"2024-04-25T21:04:57.617104+0000","last_active":"2024-04-25T21:08:28.360443+0000","last_peered":"2024-04-25T21:08:28.360443+0000","last_clean":"2024-04-25T21:08:28.360443+0000","last_became_active":"2024-04-25T21:04:57.616439+0000","last_became_peered":"2024-04-25T21:04:57.616439+0000","last_unstale":"2024-04-25T21:08:28.360443+0000","last_undegraded":"2024-04-25T21:08:28.360443+0000","last_fullsized":"2024-04-25T21:08:28.360443+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:04:56.598649+0000","last_clean_scrub_stamp":"2024-04-25T21:04:56.598649+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:44:06.548088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.358180+0000","last_change":"2024-04-25T21:05:01.653969+0000","last_active":"2024-04-25T21:08:28.358180+0000","last_peered":"2024-04-25T21:08:28.358180+0000","last_clean":"2024-04-25T21:08:28.358180+0000","last_became_active":"2024-04-25T21:05:01.653339+0000","last_became_peered":"2024-04-25T21:05:01.653339+0000","last_unstale":"2024-04-25T21:08:28.358180+0000","last_undegraded":"2024-04-25T21:08:28.358180+0000","last_fullsized":"2024-04-25T21:08:28.358180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:05.670301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362227+0000","last_change":"2024-04-25T21:05:35.848543+0000","last_active":"2024-04-25T21:08:28.362227+0000","last_peered":"2024-04-25T21:08:28.362227+0000","last_clean":"2024-04-25T21:08:28.362227+0000","last_became_active":"2024-04-25T21:05:35.848146+0000","last_became_peered":"2024-04-25T21:05:35.848146+0000","last_unstale":"2024-04-25T21:08:28.362227+0000","last_undegraded":"2024-04-25T21:08:28.362227+0000","last_fullsized":"2024-04-25T21:08:28.362227+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.203505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100956+0000","last_change":"2024-04-25T21:05:35.814755+0000","last_active":"2024-04-25T21:08:28.100956+0000","last_peered":"2024-04-25T21:08:28.100956+0000","last_clean":"2024-04-25T21:08:28.100956+0000","last_became_active":"2024-04-25T21:05:35.814385+0000","last_became_peered":"2024-04-25T21:05:35.814385+0000","last_unstale":"2024-04-25T21:08:28.100956+0000","last_undegraded":"2024-04-25T21:08:28.100956+0000","last_fullsized":"2024-04-25T21:08:28.100956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:22.263385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100859+0000","last_change":"2024-04-25T21:05:01.668424+0000","last_active":"2024-04-25T21:08:28.100859+0000","last_peered":"2024-04-25T21:08:28.100859+0000","last_clean":"2024-04-25T21:08:28.100859+0000","last_became_active":"2024-04-25T21:05:01.668167+0000","last_became_peered":"2024-04-25T21:05:01.668167+0000","last_unstale":"2024-04-25T21:08:28.100859+0000","last_undegraded":"2024-04-25T21:08:28.100859+0000","last_fullsized":"2024-04-25T21:08:28.100859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:56.086926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362713+0000","last_change":"2024-04-25T21:05:01.668335+0000","last_active":"2024-04-25T21:08:28.362713+0000","last_peered":"2024-04-25T21:08:28.362713+0000","last_clean":"2024-04-25T21:08:28.362713+0000","last_became_active":"2024-04-25T21:05:01.668007+0000","last_became_peered":"2024-04-25T21:05:01.668007+0000","last_unstale":"2024-04-25T21:08:28.362713+0000","last_undegraded":"2024-04-25T21:08:28.362713+0000","last_fullsized":"2024-04-25T21:08:28.362713+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:14.430195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360781+0000","last_change":"2024-04-25T21:05:01.663122+0000","last_active":"2024-04-25T21:08:28.360781+0000","last_peered":"2024-04-25T21:08:28.360781+0000","last_clean":"2024-04-25T21:08:28.360781+0000","last_became_active":"2024-04-25T21:05:01.662258+0000","last_became_peered":"2024-04-25T21:05:01.662258+0000","last_unstale":"2024-04-25T21:08:28.360781+0000","last_undegraded":"2024-04-25T21:08:28.360781+0000","last_fullsized":"2024-04-25T21:08:28.360781+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:26.862546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362381+0000","last_change":"2024-04-25T21:05:35.848642+0000","last_active":"2024-04-25T21:08:28.362381+0000","last_peered":"2024-04-25T21:08:28.362381+0000","last_clean":"2024-04-25T21:08:28.362381+0000","last_became_active":"2024-04-25T21:05:35.848201+0000","last_became_peered":"2024-04-25T21:05:35.848201+0000","last_unstale":"2024-04-25T21:08:28.362381+0000","last_undegraded":"2024-04-25T21:08:28.362381+0000","last_fullsized":"2024-04-25T21:08:28.362381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:05.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.101477+0000","last_change":"2024-04-25T21:05:01.650628+0000","last_active":"2024-04-25T21:08:28.101477+0000","last_peered":"2024-04-25T21:08:28.101477+0000","last_clean":"2024-04-25T21:08:28.101477+0000","last_became_active":"2024-04-25T21:05:01.650303+0000","last_became_peered":"2024-04-25T21:05:01.650303+0000","last_unstale":"2024-04-25T21:08:28.101477+0000","last_undegraded":"2024-04-25T21:08:28.101477+0000","last_fullsized":"2024-04-25T21:08:28.101477+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:20:37.967514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.195449+0000","last_change":"2024-04-25T21:05:01.655697+0000","last_active":"2024-04-25T21:08:28.195449+0000","last_peered":"2024-04-25T21:08:28.195449+0000","last_clean":"2024-04-25T21:08:28.195449+0000","last_became_active":"2024-04-25T21:05:01.655381+0000","last_became_peered":"2024-04-25T21:05:01.655381+0000","last_unstale":"2024-04-25T21:08:28.195449+0000","last_undegraded":"2024-04-25T21:08:28.195449+0000","last_fullsized":"2024-04-25T21:08:28.195449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:00.666169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.202804+0000","last_change":"2024-04-25T21:05:01.652162+0000","last_active":"2024-04-25T21:08:28.202804+0000","last_peered":"2024-04-25T21:08:28.202804+0000","last_clean":"2024-04-25T21:08:28.202804+0000","last_became_active":"2024-04-25T21:05:01.649378+0000","last_became_peered":"2024-04-25T21:05:01.649378+0000","last_unstale":"2024-04-25T21:08:28.202804+0000","last_undegraded":"2024-04-25T21:08:28.202804+0000","last_fullsized":"2024-04-25T21:08:28.202804+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:44.737375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.194954+0000","last_change":"2024-04-25T21:05:01.654379+0000","last_active":"2024-04-25T21:08:28.194954+0000","last_peered":"2024-04-25T21:08:28.194954+0000","last_clean":"2024-04-25T21:08:28.194954+0000","last_became_active":"2024-04-25T21:05:01.650322+0000","last_became_peered":"2024-04-25T21:05:01.650322+0000","last_unstale":"2024-04-25T21:08:28.194954+0000","last_undegraded":"2024-04-25T21:08:28.194954+0000","last_fullsized":"2024-04-25T21:08:28.194954+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:29:18.543477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.100689+0000","last_change":"2024-04-25T21:05:35.818506+0000","last_active":"2024-04-25T21:08:28.100689+0000","last_peered":"2024-04-25T21:08:28.100689+0000","last_clean":"2024-04-25T21:08:28.100689+0000","last_became_active":"2024-04-25T21:05:35.818074+0000","last_became_peered":"2024-04-25T21:05:35.818074+0000","last_unstale":"2024-04-25T21:08:28.100689+0000","last_undegraded":"2024-04-25T21:08:28.100689+0000","last_fullsized":"2024-04-25T21:08:28.100689+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:54.321320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,8],"acting":[1,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.360553+0000","last_change":"2024-04-25T21:05:01.668520+0000","last_active":"2024-04-25T21:08:28.360553+0000","last_peered":"2024-04-25T21:08:28.360553+0000","last_clean":"2024-04-25T21:08:28.360553+0000","last_became_active":"2024-04-25T21:05:01.668159+0000","last_became_peered":"2024-04-25T21:05:01.668159+0000","last_unstale":"2024-04-25T21:08:28.360553+0000","last_undegraded":"2024-04-25T21:08:28.360553+0000","last_fullsized":"2024-04-25T21:08:28.360553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:49.488662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T21:08:28.362318+0000","last_change":"2024-04-25T21:05:01.661228+0000","last_active":"2024-04-25T21:08:28.362318+0000","last_peered":"2024-04-25T21:08:28.362318+0000","last_clean":"2024-04-25T21:08:28.362318+0000","last_became_active":"2024-04-25T21:05:01.660878+0000","last_became_peered":"2024-04-25T21:05:01.660878+0000","last_unstale":"2024-04-25T21:08:28.362318+0000","last_undegraded":"2024-04-25T21:08:28.362318+0000","last_fullsized":"2024-04-25T21:08:28.362318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T21:05:00.624069+0000","last_clean_scrub_stamp":"2024-04-25T21:05:00.624069+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:48:01.847054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106938,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800973,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2337,"num_read_kb":1392,"num_write":515,"num_write_kb":2391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2402919,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":392,"ondisk_log_size":392,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":491622,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1208320,"data_stored":1478846,"data_compressed":53758,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":882564,"data_compressed":22199,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":882360,"data_compressed":22199,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332175,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083949,"data_compressed":38556,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":489456,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":559754,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":492026,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":562128,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182500,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1210338,"data_compressed":37401,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":489708,"data_compressed":2910,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757338,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1130496,"data_stored":1410686,"data_compressed":53758,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35646,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":73902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":331975,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394134,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400135,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T21:08:40.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:40.431 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:40.825 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:40 smithi037 ceph-mon[29195]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:40 smithi098 ceph-mon[32569]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:41.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:40 smithi177 ceph-mon[33043]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T21:08:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:41 smithi098 ceph-mon[32569]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T21:08:41.941 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:41 smithi037 ceph-mon[29195]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T21:08:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:41 smithi177 ceph-mon[33043]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-25T21:08:42.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 -- 172.21.15.37:0/1455118249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 msgr2=0x7effd810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 --2- 172.21.15.37:0/1455118249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd810f900 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7effd000b0a0 tx=0x7effd000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 -- 172.21.15.37:0/1455118249 shutdown_connections 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 --2- 172.21.15.37:0/1455118249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd810f900 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 --2- 172.21.15.37:0/1455118249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd8108a30 0x7effd8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 --2- 172.21.15.37:0/1455118249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd806a750 0x7effd806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.199+0000 7effdddf7700 1 -- 172.21.15.37:0/1455118249 >> 172.21.15.37:0/1455118249 conn(0x7effd80788f0 msgr2=0x7effd8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.200+0000 7effdddf7700 1 -- 172.21.15.37:0/1455118249 shutdown_connections 2024-04-25T21:08:42.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.200+0000 7effdddf7700 1 -- 172.21.15.37:0/1455118249 wait complete. 2024-04-25T21:08:42.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.200+0000 7effdddf7700 1 Processor -- start 2024-04-25T21:08:42.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.200+0000 7effdddf7700 1 -- start start 2024-04-25T21:08:42.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd806a750 0x7effd8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd8108a30 0x7effd8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112bd0 con 0x7effd806b0f0 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112a50 con 0x7effd8108a30 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effdddf7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7effd8112d50 con 0x7effd806a750 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36764/0 (socket says 172.21.15.37:36764) 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 -- 172.21.15.37:0/168105744 learned_addr learned my addr 172.21.15.37:0/168105744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd7fff700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd8108a30 0x7effd8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd77fe700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd806a750 0x7effd8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 -- 172.21.15.37:0/168105744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd806a750 msgr2=0x7effd8101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd806a750 0x7effd8101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 -- 172.21.15.37:0/168105744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd8108a30 msgr2=0x7effd8102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd8108a30 0x7effd8102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 -- 172.21.15.37:0/168105744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd81a7640 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd6ffd700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c920 tx=0x7effcc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.201+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc007b00 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.202+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd81a7930 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.202+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc004ce0 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.202+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc005550 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.202+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd81a7e70 con 0x7effd806b0f0 2024-04-25T21:08:42.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.202+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd8069730 con 0x7effd806b0f0 2024-04-25T21:08:42.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.204+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7effcc007ca0 con 0x7effd806b0f0 2024-04-25T21:08:42.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.204+0000 7effd4ff9700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 0x7effc00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.204+0000 7effd77fe700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 0x7effc00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.204+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7effcc0d0ac0 con 0x7effd806b0f0 2024-04-25T21:08:42.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.205+0000 7effd77fe700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 0x7effc00a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7effc800bbb0 tx=0x7effc800b590 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:42.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.208+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effcc014030 con 0x7effd806b0f0 2024-04-25T21:08:42.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.465+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd802df80 con 0x7effd806b0f0 2024-04-25T21:08:42.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.466+0000 7effd4ff9700 1 -- 172.21.15.37:0/168105744 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7effcc09cf40 con 0x7effd806b0f0 2024-04-25T21:08:42.467 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:42.467 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.468+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 msgr2=0x7effc00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.468+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 0x7effc00a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7effc800bbb0 tx=0x7effc800b590 comp rx=0 tx=0).stop 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.468+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 msgr2=0x7effd8101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.468+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7effcc00c920 tx=0x7effcc00cc30 comp rx=0 tx=0).stop 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 shutdown_connections 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7effc00a3ed0 0x7effc00a6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effd8108a30 0x7effd8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effd806a750 0x7effd8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 --2- 172.21.15.37:0/168105744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd806b0f0 0x7effd8101f50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 >> 172.21.15.37:0/168105744 conn(0x7effd80788f0 msgr2=0x7effd80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 shutdown_connections 2024-04-25T21:08:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:42.469+0000 7effdddf7700 1 -- 172.21.15.37:0/168105744 wait complete. 2024-04-25T21:08:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:08:42.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:42 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/168105744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:43.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:42 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/168105744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:43.162 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:43.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:42 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/168105744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:43.786 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:43 smithi037 ceph-mon[29195]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T21:08:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 -- 172.21.15.37:0/1633958089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 msgr2=0x7f489010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 --2- 172.21.15.37:0/1633958089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f489010ee60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4888009ca0 tx=0x7f488802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 -- 172.21.15.37:0/1633958089 shutdown_connections 2024-04-25T21:08:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 --2- 172.21.15.37:0/1633958089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f489010ee60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 --2- 172.21.15.37:0/1633958089 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4890102b70 0x7f4890106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 --2- 172.21.15.37:0/1633958089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48901021c0 0x7f48901025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 -- 172.21.15.37:0/1633958089 >> 172.21.15.37:0/1633958089 conn(0x7f48900fda20 msgr2=0x7f48900ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 -- 172.21.15.37:0/1633958089 shutdown_connections 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.956+0000 7f48974cb640 1 -- 172.21.15.37:0/1633958089 wait complete. 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 Processor -- start 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 -- start start 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 0x7f4890198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 0x7f489019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48901122a0 con 0x7f4890107290 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4890112120 con 0x7f489019cee0 2024-04-25T21:08:43.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f48974cb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4890112420 con 0x7f48901021c0 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36774/0 (socket says 172.21.15.37:36774) 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 -- 172.21.15.37:0/1991133741 learned_addr learned my addr 172.21.15.37:0/1991133741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4895267640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 0x7f4890198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4895a68640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 0x7f489019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 -- 172.21.15.37:0/1991133741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 msgr2=0x7f4890198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 0x7f4890198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 -- 172.21.15.37:0/1991133741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 msgr2=0x7f489019d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 0x7f489019d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 -- 172.21.15.37:0/1991133741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489019d9d0 con 0x7f4890107290 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4895267640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 0x7f4890198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4895a68640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 0x7f489019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.957+0000 7f4894a66640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f488000b450 tx=0x7f488000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.958+0000 7f48867fc640 1 -- 172.21.15.37:0/1991133741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880015020 con 0x7f4890107290 2024-04-25T21:08:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.958+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489019dcc0 con 0x7f4890107290 2024-04-25T21:08:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.958+0000 7f48867fc640 1 -- 172.21.15.37:0/1991133741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4880011d40 con 0x7f4890107290 2024-04-25T21:08:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.958+0000 7f48867fc640 1 -- 172.21.15.37:0/1991133741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488000f340 con 0x7f4890107290 2024-04-25T21:08:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.958+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a54b0 con 0x7f4890107290 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f48867fc640 1 -- 172.21.15.37:0/1991133741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4880018020 con 0x7f4890107290 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 Processor -- start 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 -- start start 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x559327001900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x5593270106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559326f66970 con 0x559327010c10 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559326f034a0 con 0x559327001e40 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f489a1d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559326f62930 con 0x559327001520 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36788/0 (socket says 172.21.15.37:36788) 2024-04-25T21:08:43.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f4894a66640 1 -- 172.21.15.37:0/1008797690 learned_addr learned my addr 172.21.15.37:0/1008797690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f48867fc640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 0x7f487c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f4895a68640 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x559327001900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.961+0000 7f4895267640 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x5593270106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 -- 172.21.15.37:0/1008797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 msgr2=0x559327001900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x559327001900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 -- 172.21.15.37:0/1008797690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 msgr2=0x5593270106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x5593270106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4895a68640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 0x7f487c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f48867fc640 1 -- 172.21.15.37:0/1991133741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f48800d04c0 con 0x7f4890107290 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 -- 172.21.15.37:0/1008797690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559326c79210 con 0x559327010c10 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f4894a66640 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f48800cdc60 tx=0x7f4880051490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b0c36285152e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f487b7fe640 1 -- 172.21.15.37:0/1008797690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488009ca50 con 0x559327010c10 2024-04-25T21:08:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f487b7fe640 1 -- 172.21.15.37:0/1008797690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488009c350 con 0x559327010c10 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.962+0000 7f487b7fe640 1 -- 172.21.15.37:0/1008797690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880010500 con 0x559327010c10 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- 172.21.15.37:0/1008797690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 msgr2=0x559327013000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f4895a68640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 0x7f487c0a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4888009430 tx=0x7f48880095c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f48800cdc60 tx=0x7f4880051490 comp rx=0 tx=0).stop 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- 172.21.15.37:0/1008797690 shutdown_connections 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013000 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x5593270106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 --2- 172.21.15.37:0/1008797690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x559327001900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- 172.21.15.37:0/1008797690 >> 172.21.15.37:0/1008797690 conn(0x559326ffdb30 msgr2=0x559326ffdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- 172.21.15.37:0/1008797690 shutdown_connections 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- 172.21.15.37:0/1008797690 wait complete. 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 -- start start 2024-04-25T21:08:43.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.963+0000 7f489a1d7740 1 Processor -- start 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x5593270104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x55932700d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5593270137e0 con 0x559327010c10 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559326f010b0 con 0x559327001e40 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559327013960 con 0x559327001520 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f489a1d7740 10 client.0 _ll_register_callbacks cb 0x559326eeed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36804/0 (socket says 172.21.15.37:36804) 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 -- 172.21.15.37:0/2203322585 learned_addr learned my addr 172.21.15.37:0/2203322585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4894a66640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x55932700d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895267640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x5593270104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 -- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 msgr2=0x5593270104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x5593270104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 -- 172.21.15.37:0/2203322585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 msgr2=0x55932700d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x55932700d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 -- 172.21.15.37:0/2203322585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559326d45620 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4895a68640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4888035000 tx=0x7f488802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c6b2af18dfdda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.964+0000 7f4878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888004450 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f48880045f0 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488009c350 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f48880047f0 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 10 client.15321 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55932701f6a0 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488009c350 con 0x559327010c10 2024-04-25T21:08:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f48880306e0 con 0x559327010c10 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 10 client.15321 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 20 client.15321 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327020ba0 0x559327022fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 4 client.15321 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559326f74390 con 0x559327020ba0 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.965+0000 7f4895267640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327020ba0 0x559327022fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.966+0000 7f4895267640 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327020ba0 0x559327022fb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a25373aba069b88 server_cookie=ed41cc6640a10cfc in_seq=0 out_seq=0 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.966+0000 7f4878ff9640 10 client.15321 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.966+0000 7f4878ff9640 1 -- 172.21.15.37:0/2203322585 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559326f74390 con 0x559327020ba0 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f4878ff9640 10 client.15321 handle_command_reply: tid=0 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 client.15321 shutdown 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 msgr2=0x55932700dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4888035000 tx=0x7f488802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 shutdown_connections 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327020ba0 0x559327022fb0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x55932700dcd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001e40 0x55932700d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2203322585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x5593270104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 >> 172.21.15.37:0/2203322585 conn(0x559326ffdb30 msgr2=0x559327003970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f4878ff9640 0 client.15321 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 shutdown_connections 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 1 -- 172.21.15.37:0/2203322585 wait complete. 2024-04-25T21:08:43.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.967+0000 7f489a1d7740 20 client.15321 trim_cache size 0 max 16384 2024-04-25T21:08:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 Processor -- start 2024-04-25T21:08:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 -- start start 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x55932701c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 0x55932700ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ab780 con 0x559327001e40 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ad2c0 con 0x559327010c10 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f489a1d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48901abe50 con 0x559327001520 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.982+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36814/0 (socket says 172.21.15.37:36814) 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 -- 172.21.15.37:0/391841178 learned_addr learned my addr 172.21.15.37:0/391841178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4895267640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x55932701c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4895a68640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 0x55932700ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 -- 172.21.15.37:0/391841178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 msgr2=0x55932701c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x55932701c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 -- 172.21.15.37:0/391841178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 msgr2=0x55932700ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 0x55932700ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 -- 172.21.15.37:0/391841178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559326ff92e0 con 0x559327001e40 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4895267640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x55932701c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4895a68640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 0x55932700ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f4894a66640 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f488009c2e0 tx=0x7f48800989c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6104d8c18655657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f487affd640 1 -- 172.21.15.37:0/391841178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800a0700 con 0x559327001e40 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f487affd640 1 -- 172.21.15.37:0/391841178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488009c350 con 0x559327001e40 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.983+0000 7f487affd640 1 -- 172.21.15.37:0/391841178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48800172c0 con 0x559327001e40 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- 172.21.15.37:0/391841178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 msgr2=0x559327008d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f488009c2e0 tx=0x7f48800989c0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- 172.21.15.37:0/391841178 shutdown_connections 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327010c10 0x55932700ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327001e40 0x559327008d80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 --2- 172.21.15.37:0/391841178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001520 0x55932701c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- 172.21.15.37:0/391841178 >> 172.21.15.37:0/391841178 conn(0x559326ffdb30 msgr2=0x559327008670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- 172.21.15.37:0/391841178 shutdown_connections 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- 172.21.15.37:0/391841178 wait complete. 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 -- start start 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.984+0000 7f489a1d7740 1 Processor -- start 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 0x559327018b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001e40 0x5593270190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f48901acbf0 con 0x559327010c10 2024-04-25T21:08:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f48901ac520 con 0x559327001520 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f48901a9760 con 0x559327001e40 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 10 client.0 _ll_register_callbacks cb 0x7f48880318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36826/0 (socket says 172.21.15.37:36826) 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 -- 172.21.15.37:0/2248308972 learned_addr learned my addr 172.21.15.37:0/2248308972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895267640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 0x559327018b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4894a66640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001e40 0x5593270190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 -- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001e40 msgr2=0x5593270190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001e40 0x5593270190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 -- 172.21.15.37:0/2248308972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 msgr2=0x559327018b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 0x559327018b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 -- 172.21.15.37:0/2248308972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559327001c70 con 0x559327010c10 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895267640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 0x559327018b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4895a68640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4888030d10 tx=0x7f4888005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db9b2dd1e356cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48880305c0 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4888030760 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488009c350 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4888004250 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.985+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559327086ad0 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f488009c350 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4888061030 con 0x559327010c10 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 20 client.15327 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327087140 0x559327089530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 4 client.15327 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559326f6e200 con 0x559327087140 2024-04-25T21:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.986+0000 7f4895267640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327087140 0x559327089530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.987+0000 7f4895267640 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327087140 0x559327089530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77808a75523b4d6 server_cookie=7185430301d1a48e in_seq=0 out_seq=0 2024-04-25T21:08:43.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:43.987+0000 7f4878ff9640 10 client.15327 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f4878ff9640 1 -- 172.21.15.37:0/2248308972 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559326f6e200 con 0x559327087140 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f4878ff9640 10 client.15327 handle_command_reply: tid=0 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 client.15327 shutdown 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 msgr2=0x559327013fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4888030d10 tx=0x7f4888005f70 comp rx=0 tx=0).stop 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 shutdown_connections 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559327087140 0x559327089530 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559327010c10 0x559327013fd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f4878ff9640 0 client.15327 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559327001e40 0x5593270190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 --2- 172.21.15.37:0/2248308972 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559327001520 0x559327018b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 >> 172.21.15.37:0/2248308972 conn(0x559326ffdb30 msgr2=0x559327003070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 shutdown_connections 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 1 -- 172.21.15.37:0/2248308972 wait complete. 2024-04-25T21:08:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.000+0000 7f489a1d7740 20 client.15327 trim_cache size 0 max 16384 2024-04-25T21:08:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:08:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:08:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:08:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 msgr2=0x7f487c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 0x7f487c0a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4888009430 tx=0x7f48880095c0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 msgr2=0x7f4890198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f488000b450 tx=0x7f488000b910 comp rx=0 tx=0).stop 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 shutdown_connections 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f489019cee0 0x7f489019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4890107290 0x7f4890198b60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f487c0a3ee0 0x7f487c0a63a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 --2- 172.21.15.37:0/1991133741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48901021c0 0x7f4890198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 >> 172.21.15.37:0/1991133741 conn(0x7f48900fda20 msgr2=0x7f48900fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 shutdown_connections 2024-04-25T21:08:44.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:44.001+0000 7f48974cb640 1 -- 172.21.15.37:0/1991133741 wait complete. 2024-04-25T21:08:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:43 smithi177 ceph-mon[33043]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T21:08:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:43 smithi098 ceph-mon[32569]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 -- 172.21.15.37:0/3998148552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908103450 msgr2=0x7fb90810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 --2- 172.21.15.37:0/3998148552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908103450 0x7fb90810f8e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb90000b0a0 tx=0x7fb90000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 -- 172.21.15.37:0/3998148552 shutdown_connections 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 --2- 172.21.15.37:0/3998148552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908103450 0x7fb90810f8e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 --2- 172.21.15.37:0/3998148552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 0x7fb908108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 --2- 172.21.15.37:0/3998148552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908102a90 0x7fb908102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 -- 172.21.15.37:0/3998148552 >> 172.21.15.37:0/3998148552 conn(0x7fb908079bf0 msgr2=0x7fb90807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.139+0000 7fb90f048700 1 -- 172.21.15.37:0/3998148552 shutdown_connections 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 -- 172.21.15.37:0/3998148552 wait complete. 2024-04-25T21:08:45.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 Processor -- start 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 -- start start 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908103450 0x7fb908075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 0x7fb9080757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb908112ce0 con 0x7fb908102a90 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb908112b60 con 0x7fb908103450 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.140+0000 7fb90f048700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb908112e60 con 0x7fb908108a80 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36842/0 (socket says 172.21.15.37:36842) 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 -- 172.21.15.37:0/3185761345 learned_addr learned my addr 172.21.15.37:0/3185761345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90d5e5700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 0x7fb9080757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb907fff700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908103450 0x7fb908075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 -- 172.21.15.37:0/3185761345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 msgr2=0x7fb9080757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:45.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 0x7fb9080757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 -- 172.21.15.37:0/3185761345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908103450 msgr2=0x7fb908075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908103450 0x7fb908075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 -- 172.21.15.37:0/3185761345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9081a76b0 con 0x7fb908102a90 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90cde4700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b530 tx=0x7fb8f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8015020 con 0x7fb908102a90 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9081a79a0 con 0x7fb908102a90 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8f8011ce0 con 0x7fb908102a90 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.141+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8010e40 con 0x7fb908102a90 2024-04-25T21:08:45.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.142+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9081a7e90 con 0x7fb908102a90 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.143+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb908069730 con 0x7fb908102a90 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.144+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb8f8018020 con 0x7fb908102a90 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.144+0000 7fb905ffb700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 0x7fb8f00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.144+0000 7fb907fff700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 0x7fb8f00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.144+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb8f80d00a0 con 0x7fb908102a90 2024-04-25T21:08:45.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.145+0000 7fb907fff700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 0x7fb8f00a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc00a560 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.149+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8f801f020 con 0x7fb908102a90 2024-04-25T21:08:45.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.405+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb90802df80 con 0x7fb908102a90 2024-04-25T21:08:45.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.405+0000 7fb905ffb700 1 -- 172.21.15.37:0/3185761345 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fb8f809c7a0 con 0x7fb908102a90 2024-04-25T21:08:45.406 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:45.406 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 msgr2=0x7fb8f00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 0x7fb8f00a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc00a560 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 msgr2=0x7fb908074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800b530 tx=0x7fb8f800b840 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 shutdown_connections 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8f00a3ed0 0x7fb8f00a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb908108a80 0x7fb9080757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb908102a90 0x7fb908074d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 --2- 172.21.15.37:0/3185761345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb908103450 0x7fb908075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 >> 172.21.15.37:0/3185761345 conn(0x7fb908079bf0 msgr2=0x7fb9081016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.408+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 shutdown_connections 2024-04-25T21:08:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:45.409+0000 7fb90f048700 1 -- 172.21.15.37:0/3185761345 wait complete. 2024-04-25T21:08:45.410 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:08:45.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:45 smithi098 ceph-mon[32569]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-04-25T21:08:45.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3185761345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:45 smithi037 ceph-mon[29195]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-04-25T21:08:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3185761345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:46.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:45 smithi177 ceph-mon[33043]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-04-25T21:08:46.111 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3185761345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:46.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:08:46.378 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:47.784 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:47 smithi037 ceph-mon[29195]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 KiB/s wr, 26 op/s 2024-04-25T21:08:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:47 smithi177 ceph-mon[33043]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 KiB/s wr, 26 op/s 2024-04-25T21:08:48.164 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:47 smithi098 ceph-mon[32569]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.5 KiB/s wr, 26 op/s 2024-04-25T21:08:48.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 -- 172.21.15.37:0/4116004218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464108a50 msgr2=0x7fa464108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:48.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 --2- 172.21.15.37:0/4116004218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464108a50 0x7fa464108e30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa454009990 tx=0x7fa454009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 -- 172.21.15.37:0/4116004218 shutdown_connections 2024-04-25T21:08:48.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 --2- 172.21.15.37:0/4116004218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464103450 0x7fa46410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 --2- 172.21.15.37:0/4116004218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464108a50 0x7fa464108e30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 --2- 172.21.15.37:0/4116004218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464102a90 0x7fa464102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 -- 172.21.15.37:0/4116004218 >> 172.21.15.37:0/4116004218 conn(0x7fa464079ba0 msgr2=0x7fa464079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:48.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 -- 172.21.15.37:0/4116004218 shutdown_connections 2024-04-25T21:08:48.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 -- 172.21.15.37:0/4116004218 wait complete. 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.170+0000 7fa469a52700 1 Processor -- start 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 -- start start 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464102a90 0x7fa464199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464108a50 0x7fa46419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa464112c00 con 0x7fa464103450 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa464112a80 con 0x7fa464108a50 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa469a52700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa464112d80 con 0x7fa464102a90 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:48.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36856/0 (socket says 172.21.15.37:36856) 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 -- 172.21.15.37:0/2691147494 learned_addr learned my addr 172.21.15.37:0/2691147494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa462ffd700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464102a90 0x7fa464199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4637fe700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464108a50 0x7fa46419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 -- 172.21.15.37:0/2691147494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464102a90 msgr2=0x7fa464199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464102a90 0x7fa464199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 -- 172.21.15.37:0/2691147494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464108a50 msgr2=0x7fa46419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464108a50 0x7fa46419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 -- 172.21.15.37:0/2691147494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46419e120 con 0x7fa464103450 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.171+0000 7fa4627fc700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45800e9a0 tx=0x7fa45800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.172+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45800ca10 con 0x7fa464103450 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.172+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa45800cbb0 con 0x7fa464103450 2024-04-25T21:08:48.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.172+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46419e410 con 0x7fa464103450 2024-04-25T21:08:48.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.172+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa458018680 con 0x7fa464103450 2024-04-25T21:08:48.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.172+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4641a5c90 con 0x7fa464103450 2024-04-25T21:08:48.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.173+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa464069730 con 0x7fa464103450 2024-04-25T21:08:48.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.174+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa458010430 con 0x7fa464103450 2024-04-25T21:08:48.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.175+0000 7fa468a50700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 0x7fa44c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:48.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.175+0000 7fa462ffd700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 0x7fa44c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:48.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.175+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa4580d0f40 con 0x7fa464103450 2024-04-25T21:08:48.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.175+0000 7fa462ffd700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 0x7fa44c0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b2e0 tx=0x7fa454005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:48.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.178+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa45809d360 con 0x7fa464103450 2024-04-25T21:08:48.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.389+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa46419a470 con 0x7fa464103450 2024-04-25T21:08:48.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.391+0000 7fa468a50700 1 -- 172.21.15.37:0/2691147494 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22948 (secure 0 0 0) 0x7fa4580bbe20 con 0x7fa464103450 2024-04-25T21:08:48.391 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:48.391 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:08:28.094274+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/1634118111":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/3727603400":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/234140491":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1398754476":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:08:48.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 msgr2=0x7fa44c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:48.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 0x7fa44c0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa45400b2e0 tx=0x7fa454005960 comp rx=0 tx=0).stop 2024-04-25T21:08:48.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 msgr2=0x7fa464199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:48.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa45800e9a0 tx=0x7fa45800ecb0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 shutdown_connections 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa44c0a3ed0 0x7fa44c0a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa464108a50 0x7fa46419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa464102a90 0x7fa464199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 --2- 172.21.15.37:0/2691147494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa464103450 0x7fa464199650 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 >> 172.21.15.37:0/2691147494 conn(0x7fa464079ba0 msgr2=0x7fa4641018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 shutdown_connections 2024-04-25T21:08:48.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:48.393+0000 7fa469a52700 1 -- 172.21.15.37:0/2691147494 wait complete. 2024-04-25T21:08:48.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2691147494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:49.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2691147494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:49.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T21:08:49.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2691147494' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:49.351 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:49.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:49 smithi037 ceph-mon[29195]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:49 smithi177 ceph-mon[33043]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:49 smithi098 ceph-mon[32569]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:51.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:51 smithi037 ceph-mon[29195]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-04-25T21:08:52.015 INFO:teuthology.orchestra.run.smithi037.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T21:08:52.016 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T21:08:52.016 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2919 2024-04-25T21:08:52.016 DEBUG:tasks.quiescer:Yielding 2024-04-25T21:08:52.016 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T21:08:52.030 INFO:tasks.workunit:Pulling workunits from ref e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T21:08:52.031 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T21:08:52.031 INFO:tasks.workunit:timeout=3h 2024-04-25T21:08:52.031 INFO:tasks.workunit:cleanup=True 2024-04-25T21:08:52.031 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:52.032 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:52.048 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 21:05:35.744533312 +0000 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 21:05:35.744533312 +0000 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 21:08:25.792649730 +0000 2024-04-25T21:08:52.049 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-25 21:05:35.744533312 +0000 2024-04-25T21:08:52.049 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:52.049 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T21:08:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:51 smithi177 ceph-mon[33043]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-04-25T21:08:52.122 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T21:08:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:51 smithi098 ceph-mon[32569]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-04-25T21:08:52.178 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T21:08:52.291 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:53.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:53 smithi037 ceph-mon[29195]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:54.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:53 smithi177 ceph-mon[33043]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:54.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 -- 172.21.15.37:0/531799646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4101940 msgr2=0x7fdac4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 --2- 172.21.15.37:0/531799646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4101940 0x7fdac4101dc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009990 tx=0x7fdab0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 -- 172.21.15.37:0/531799646 shutdown_connections 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 --2- 172.21.15.37:0/531799646 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 0x7fdac410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 --2- 172.21.15.37:0/531799646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4107990 0x7fdac4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 --2- 172.21.15.37:0/531799646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4101940 0x7fdac4101dc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 -- 172.21.15.37:0/531799646 >> 172.21.15.37:0/531799646 conn(0x7fdac40788f0 msgr2=0x7fdac4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 -- 172.21.15.37:0/531799646 shutdown_connections 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.139+0000 7fdacabbd700 1 -- 172.21.15.37:0/531799646 wait complete. 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 Processor -- start 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 -- start start 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4101940 0x7fdac4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:54.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 0x7fdac41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4111b50 con 0x7fdac4107990 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdac41119d0 con 0x7fdac4102300 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdacabbd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdac4111cd0 con 0x7fdac4101940 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50752/0 (socket says 172.21.15.37:50752) 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 -- 172.21.15.37:0/698796936 learned_addr learned my addr 172.21.15.37:0/698796936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac8959700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4101940 0x7fdac4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac3fff700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 0x7fdac41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 -- 172.21.15.37:0/698796936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4101940 msgr2=0x7fdac4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4101940 0x7fdac4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 -- 172.21.15.37:0/698796936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 msgr2=0x7fdac41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 0x7fdac41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.140+0000 7fdac915a700 1 -- 172.21.15.37:0/698796936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac419e1a0 con 0x7fdac4107990 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdac915a700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ada0 tx=0x7fdabc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc017020 con 0x7fdac4107990 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc00cd80 con 0x7fdac4107990 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac419e490 con 0x7fdac4107990 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc005260 con 0x7fdac4107990 2024-04-25T21:08:54.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.141+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac41a5d10 con 0x7fdac4107990 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.142+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac4069730 con 0x7fdac4107990 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.143+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdabc00a090 con 0x7fdac4107990 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.144+0000 7fdac1ffb700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 0x7fdab40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.144+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fdabc0d0760 con 0x7fdac4107990 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.144+0000 7fdac8959700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 0x7fdab40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:54.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.145+0000 7fdac8959700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 0x7fdab40a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009be0 tx=0x7fdab8009260 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:54.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.148+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdabc021020 con 0x7fdac4107990 2024-04-25T21:08:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:53 smithi098 ceph-mon[32569]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-25T21:08:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.412+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdac41a5ee0 con 0x7fdac4107990 2024-04-25T21:08:54.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.413+0000 7fdac1ffb700 1 -- 172.21.15.37:0/698796936 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fdabc01a310 con 0x7fdac4107990 2024-04-25T21:08:54.414 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:08:54.414 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.415+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 msgr2=0x7fdab40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 0x7fdab40a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009be0 tx=0x7fdab8009260 comp rx=0 tx=0).stop 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 msgr2=0x7fdac419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00ada0 tx=0x7fdabc00c330 comp rx=0 tx=0).stop 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 shutdown_connections 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdab40a3ed0 0x7fdab40a6390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdac4107990 0x7fdac419da60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdac4101940 0x7fdac4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 --2- 172.21.15.37:0/698796936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdac4102300 0x7fdac41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 >> 172.21.15.37:0/698796936 conn(0x7fdac40788f0 msgr2=0x7fdac4105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 shutdown_connections 2024-04-25T21:08:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:54.416+0000 7fdacabbd700 1 -- 172.21.15.37:0/698796936 wait complete. 2024-04-25T21:08:54.417 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:08:54.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:54 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/698796936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:55.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:54 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/698796936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:55.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:54 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/698796936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:55.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T21:05:03.820512+0000', 'modified': '2024-04-25T21:08:26.488484+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24469, 'mds_1': 24458, 'mds_2': 24457, 'mds_3': 24445, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14568': {'gid': 14568, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6839/2484283883', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6838', 'nonce': 2484283883}, {'type': 'v1', 'addr': '172.21.15.37:6839', 'nonce': 2484283883}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14574': {'gid': 14574, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.37:6841/2041063205', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6840', 'nonce': 2041063205}, {'type': 'v1', 'addr': '172.21.15.37:6841', 'nonce': 2041063205}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24380': {'gid': 24380, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6833/3425764534', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6832', 'nonce': 3425764534}, {'type': 'v1', 'addr': '172.21.15.98:6833', 'nonce': 3425764534}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.98:6835/785792135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 785792135}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 785792135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6833/3715259407', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6832', 'nonce': 3715259407}, {'type': 'v1', 'addr': '172.21.15.177:6833', 'nonce': 3715259407}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.98:6837/2446670971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 2446670971}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 2446670971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.177:6835/724905952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 724905952}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 724905952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.177:6837/2828024557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 2828024557}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 2828024557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'b', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.98:6839/899178427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6838', 'nonce': 899178427}, {'type': 'v1', 'addr': '172.21.15.98:6839', 'nonce': 899178427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.177:6839/3063347892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 3063347892}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 3063347892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24469, 'qdb_cluster': [24419, 24445, 24457, 24458, 24469]} 2024-04-25T21:08:55.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T21:08:55.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T21:08:55.500 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:55.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:55 smithi037 ceph-mon[29195]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-25T21:08:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:55 smithi177 ceph-mon[33043]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-25T21:08:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:55 smithi098 ceph-mon[32569]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-25T21:08:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- 172.21.15.37:0/3253512842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb154107280 msgr2=0x7fb15410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 --2- 172.21.15.37:0/3253512842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb154107280 0x7fb15410ee50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009ca0 tx=0x7fb14c02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- 172.21.15.37:0/3253512842 shutdown_connections 2024-04-25T21:08:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 --2- 172.21.15.37:0/3253512842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb154107280 0x7fb15410ee50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 --2- 172.21.15.37:0/3253512842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154102b60 0x7fb154106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 --2- 172.21.15.37:0/3253512842 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1541021b0 0x7fb154102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- 172.21.15.37:0/3253512842 >> 172.21.15.37:0/3253512842 conn(0x7fb1540fda10 msgr2=0x7fb1540ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- 172.21.15.37:0/3253512842 shutdown_connections 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- 172.21.15.37:0/3253512842 wait complete. 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 Processor -- start 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.096+0000 7fb159744640 1 -- start start 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154107280 0x7fb154198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 0x7fb15419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541121d0 con 0x7fb1541021b0 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb154112050 con 0x7fb154107280 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb154112350 con 0x7fb15419cf40 2024-04-25T21:08:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50770/0 (socket says 172.21.15.37:50770) 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 -- 172.21.15.37:0/4103859304 learned_addr learned my addr 172.21.15.37:0/4103859304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb14bfff640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154107280 0x7fb154198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb1537fe640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 0x7fb15419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 -- 172.21.15.37:0/4103859304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 msgr2=0x7fb15419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 0x7fb15419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 -- 172.21.15.37:0/4103859304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154107280 msgr2=0x7fb154198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154107280 0x7fb154198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 -- 172.21.15.37:0/4103859304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15419da30 con 0x7fb1541021b0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb1537fe640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 0x7fb15419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb152ffd640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb14400d950 tx=0x7fb14400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb150ff9640 1 -- 172.21.15.37:0/4103859304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144017070 con 0x7fb1541021b0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb150ff9640 1 -- 172.21.15.37:0/4103859304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb144009c30 con 0x7fb1541021b0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15419dd20 con 0x7fb1541021b0 2024-04-25T21:08:57.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.097+0000 7fb150ff9640 1 -- 172.21.15.37:0/4103859304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400f620 con 0x7fb1541021b0 2024-04-25T21:08:57.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.098+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1541a5560 con 0x7fb1541021b0 2024-04-25T21:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.100+0000 7fb150ff9640 1 -- 172.21.15.37:0/4103859304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb144010460 con 0x7fb1541021b0 2024-04-25T21:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.100+0000 7fb15c465740 1 Processor -- start 2024-04-25T21:08:57.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 -- start start 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e664bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d95e5bc970 con 0x55d95e657620 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d95e5594a0 con 0x55d95e657f40 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb15c465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d95e5b8930 con 0x55d95e662800 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50778/0 (socket says 172.21.15.37:50778) 2024-04-25T21:08:57.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 -- 172.21.15.37:0/3703919071 learned_addr learned my addr 172.21.15.37:0/3703919071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb152ffd640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb14bfff640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e664bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 -- 172.21.15.37:0/3703919071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 msgr2=0x55d95e664bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb150ff9640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 0x7fb1400a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e664bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 -- 172.21.15.37:0/3703919071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 msgr2=0x55d95e6622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb1537fe640 1 -- 172.21.15.37:0/3703919071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95e2cf210 con 0x55d95e657620 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.101+0000 7fb14bfff640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e664bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb150ff9640 1 -- 172.21.15.37:0/4103859304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb1440d0640 con 0x7fb1541021b0 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb1537fe640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 0x7fb1400a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0098e0 tx=0x7fb14c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf783afbdf3580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb1497fa640 1 -- 172.21.15.37:0/3703919071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c04c070 con 0x55d95e657620 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb1497fa640 1 -- 172.21.15.37:0/3703919071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb14c036070 con 0x55d95e657620 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.102+0000 7fb1497fa640 1 -- 172.21.15.37:0/3703919071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c02fba0 con 0x55d95e657620 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- 172.21.15.37:0/3703919071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 msgr2=0x55d95e657a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0098e0 tx=0x7fb14c009910 comp rx=0 tx=0).stop 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb1537fe640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 0x7fb1400a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb14c031fd0 tx=0x7fb14c04a000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- 172.21.15.37:0/3703919071 shutdown_connections 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e664bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 --2- 172.21.15.37:0/3703919071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657620 0x55d95e657a00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- 172.21.15.37:0/3703919071 >> 172.21.15.37:0/3703919071 conn(0x55d95e653b30 msgr2=0x55d95e653f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- 172.21.15.37:0/3703919071 shutdown_connections 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- 172.21.15.37:0/3703919071 wait complete. 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 -- start start 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.103+0000 7fb15c465740 1 Processor -- start 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e6621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 0x55d95e65f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d95e665430 con 0x55d95e662800 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d95e5570b0 con 0x55d95e657620 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d95e6655b0 con 0x55d95e657f40 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb15c465740 10 client.0 _ll_register_callbacks cb 0x55d95e544d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50784/0 (socket says 172.21.15.37:50784) 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 -- 172.21.15.37:0/2109775069 learned_addr learned my addr 172.21.15.37:0/2109775069 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb152ffd640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e6621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb14bfff640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 0x55d95e65f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 -- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 msgr2=0x55d95e65f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 0x55d95e65f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 -- 172.21.15.37:0/2109775069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 msgr2=0x55d95e6621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e6621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 -- 172.21.15.37:0/2109775069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95e39b620 con 0x55d95e662800 2024-04-25T21:08:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb14bfff640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 0x55d95e65f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb152ffd640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e6621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb1537fe640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb14c02fcb0 tx=0x7fb14c02f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbf93bbe1fbb193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.104+0000 7fb13a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c04c040 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb14c003fc0 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14c036070 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb14c064050 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 10 client.15357 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d95e665430 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb14c036070 con 0x55d95e662800 2024-04-25T21:08:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb14c042050 con 0x55d95e662800 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 10 client.15357 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 20 client.15357 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e672540 0x55d95e674930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 4 client.15357 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.105+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d95e5ca390 con 0x55d95e672540 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.106+0000 7fb152ffd640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e672540 0x55d95e674930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.106+0000 7fb152ffd640 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e672540 0x55d95e674930 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a930f910f79e6ca7 server_cookie=e91046a532de9be5 in_seq=0 out_seq=0 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.106+0000 7fb13a7fc640 10 client.15357 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb13a7fc640 1 -- 172.21.15.37:0/2109775069 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d95e5ca390 con 0x55d95e672540 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb13a7fc640 10 client.15357 handle_command_reply: tid=0 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 client.15357 shutdown 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 msgr2=0x55d95e65fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb14c02fcb0 tx=0x7fb14c02f880 comp rx=0 tx=0).stop 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 shutdown_connections 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e672540 0x55d95e674930 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e65fac0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657f40 0x55d95e65f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb13a7fc640 0 client.15357 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 --2- 172.21.15.37:0/2109775069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e6621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 >> 172.21.15.37:0/2109775069 conn(0x55d95e653b30 msgr2=0x55d95e6598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 shutdown_connections 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 1 -- 172.21.15.37:0/2109775069 wait complete. 2024-04-25T21:08:57.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.107+0000 7fb15c465740 20 client.15357 trim_cache size 0 max 16384 2024-04-25T21:08:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.124+0000 7fb15c465740 1 Processor -- start 2024-04-25T21:08:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.124+0000 7fb15c465740 1 -- start start 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e66e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e65a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ada80 con 0x55d95e657f40 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541af5c0 con 0x55d95e657620 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb15c465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ae150 con 0x55d95e662800 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.125+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50790/0 (socket says 172.21.15.37:50790) 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 -- 172.21.15.37:0/2704933506 learned_addr learned my addr 172.21.15.37:0/2704933506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb152ffd640 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e66e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb1537fe640 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e65a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 -- 172.21.15.37:0/2704933506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 msgr2=0x55d95e65a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e65a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 -- 172.21.15.37:0/2704933506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 msgr2=0x55d95e66e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e66e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 -- 172.21.15.37:0/2704933506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d95e674ca0 con 0x55d95e657f40 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb14bfff640 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb15406b5b0 tx=0x7fb13c00ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c621380c47aa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb148ff9640 1 -- 172.21.15.37:0/2704933506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c003870 con 0x55d95e657f40 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb148ff9640 1 -- 172.21.15.37:0/2704933506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c011440 con 0x55d95e657f40 2024-04-25T21:08:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.126+0000 7fb148ff9640 1 -- 172.21.15.37:0/2704933506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c010650 con 0x55d95e657f40 2024-04-25T21:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 -- 172.21.15.37:0/2704933506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 msgr2=0x55d95e661cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb15406b5b0 tx=0x7fb13c00ba60 comp rx=0 tx=0).stop 2024-04-25T21:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 -- 172.21.15.37:0/2704933506 shutdown_connections 2024-04-25T21:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e662800 0x55d95e65a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e657f40 0x55d95e661cf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 --2- 172.21.15.37:0/2704933506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657620 0x55d95e66e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.127+0000 7fb15c465740 1 -- 172.21.15.37:0/2704933506 >> 172.21.15.37:0/2704933506 conn(0x55d95e653b30 msgr2=0x55d95e65a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 -- 172.21.15.37:0/2704933506 shutdown_connections 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 -- 172.21.15.37:0/2704933506 wait complete. 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 -- start start 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 Processor -- start 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 0x55d95e6d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.128+0000 7fb15c465740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541aeef0 con 0x55d95e662800 2024-04-25T21:08:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb15c465740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541ae820 con 0x55d95e657f40 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb15c465740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb1541aba80 con 0x55d95e657620 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50806/0 (socket says 172.21.15.37:50806) 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 -- 172.21.15.37:0/3252994590 learned_addr learned my addr 172.21.15.37:0/3252994590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb152ffd640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 0x55d95e6d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb15c465740 10 client.0 _ll_register_callbacks cb 0x7fb14c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb14bfff640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 -- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 msgr2=0x55d95e6d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 0x55d95e6d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 -- 172.21.15.37:0/3252994590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 msgr2=0x55d95e6d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 -- 172.21.15.37:0/3252994590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95e6dbc20 con 0x55d95e662800 2024-04-25T21:08:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb152ffd640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 0x55d95e6d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb1537fe640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009430 tx=0x7fb14c031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dac6ef270b8c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb13a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c04c070 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb14c031450 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.129+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb13c011440 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb14c06f050 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 10 client.15363 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d95e6dcf10 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c011440 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb14c042040 con 0x55d95e662800 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 20 client.15363 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e6dd5d0 0x55d95e6df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 4 client.15363 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d95e5c4200 con 0x55d95e6dd5d0 2024-04-25T21:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.130+0000 7fb152ffd640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e6dd5d0 0x55d95e6df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.131+0000 7fb13a7fc640 10 client.15363 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.131+0000 7fb152ffd640 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e6dd5d0 0x55d95e6df9c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853cc29c9b28aaf3 server_cookie=e242997e7bcfd44a in_seq=0 out_seq=0 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb13a7fc640 1 -- 172.21.15.37:0/3252994590 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d95e5c4200 con 0x55d95e6dd5d0 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb13a7fc640 10 client.15363 handle_command_reply: tid=0 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 client.15363 shutdown 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 msgr2=0x55d95e6d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb14c009430 tx=0x7fb14c031bf0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 shutdown_connections 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d95e6dd5d0 0x55d95e6df9c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb13a7fc640 0 client.15363 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d95e662800 0x55d95e6d14a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d95e657f40 0x55d95e6d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 --2- 172.21.15.37:0/3252994590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d95e657620 0x55d95e6d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 >> 172.21.15.37:0/3252994590 conn(0x55d95e653b30 msgr2=0x55d95e652cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 shutdown_connections 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 1 -- 172.21.15.37:0/3252994590 wait complete. 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.140+0000 7fb15c465740 20 client.15363 trim_cache size 0 max 16384 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:08:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:08:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 msgr2=0x7fb1400a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 0x7fb1400a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb14c031fd0 tx=0x7fb14c04a000 comp rx=0 tx=0).stop 2024-04-25T21:08:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 msgr2=0x7fb1541986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb14400d950 tx=0x7fb14400de10 comp rx=0 tx=0).stop 2024-04-25T21:08:57.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 shutdown_connections 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb1400a3f60 0x7fb1400a6420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb15419cf40 0x7fb15419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb154107280 0x7fb154198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 --2- 172.21.15.37:0/4103859304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1541021b0 0x7fb1541986d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 >> 172.21.15.37:0/4103859304 conn(0x7fb1540fda10 msgr2=0x7fb1540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 shutdown_connections 2024-04-25T21:08:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:08:57.141+0000 7fb159744640 1 -- 172.21.15.37:0/4103859304 wait complete. 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 -- 172.21.15.37:0/1918880813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 msgr2=0x7f09a8102780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 --2- 172.21.15.37:0/1918880813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8102780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f099c0099c0 tx=0x7f099c009cd0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 -- 172.21.15.37:0/1918880813 shutdown_connections 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 --2- 172.21.15.37:0/1918880813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 0x7f09a810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 --2- 172.21.15.37:0/1918880813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8102780 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 --2- 172.21.15.37:0/1918880813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 0x7f09a81065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 -- 172.21.15.37:0/1918880813 >> 172.21.15.37:0/1918880813 conn(0x7f09a8077fb0 msgr2=0x7f09a80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 -- 172.21.15.37:0/1918880813 shutdown_connections 2024-04-25T21:08:57.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.372+0000 7f09af01d700 1 -- 172.21.15.37:0/1918880813 wait complete. 2024-04-25T21:08:57.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 Processor -- start 2024-04-25T21:08:57.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 -- start start 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 0x7f09a8110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 0x7f09a8111510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8112d50 con 0x7f09a81023a0 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8112bd0 con 0x7f09a8106cc0 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09af01d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8112ed0 con 0x7f09a8102d50 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09acdb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09acdb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50830/0 (socket says 172.21.15.37:50830) 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09acdb9700 1 -- 172.21.15.37:0/2116397916 learned_addr learned my addr 172.21.15.37:0/2116397916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.373+0000 7f09a7fff700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 0x7f09a8110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 0x7f09a8111510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 -- 172.21.15.37:0/2116397916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 msgr2=0x7f09a8110fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 0x7f09a8110fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 -- 172.21.15.37:0/2116397916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 msgr2=0x7f09a8111510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 0x7f09a8111510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 -- 172.21.15.37:0/2116397916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a76f0 con 0x7f09a81023a0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09acdb9700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09a5ffb700 1 -- 172.21.15.37:0/2116397916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c027070 con 0x7f09a81023a0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a7980 con 0x7f09a81023a0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09a5ffb700 1 -- 172.21.15.37:0/2116397916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c003f20 con 0x7f09a81023a0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.374+0000 7f09a5ffb700 1 -- 172.21.15.37:0/2116397916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c022ca0 con 0x7f09a81023a0 2024-04-25T21:08:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.375+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a7e30 con 0x7f09a81023a0 2024-04-25T21:08:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.377+0000 7f09a5ffb700 1 -- 172.21.15.37:0/2116397916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f099c02a070 con 0x7f09a81023a0 2024-04-25T21:08:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.378+0000 7f09a5ffb700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 0x7f09980a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.378+0000 7f09a7fff700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 0x7f09980a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.378+0000 7f09a5ffb700 1 -- 172.21.15.37:0/2116397916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f099c0d1890 con 0x7f09a81023a0 2024-04-25T21:08:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.378+0000 7f09a7fff700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 0x7f09980a6410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09a8104560 tx=0x7f099400b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 Processor -- start 2024-04-25T21:08:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 -- start start 2024-04-25T21:08:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e90503bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e905040040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e9050409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e904f8dce0 con 0x55e90503c0f0 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e904f34910 con 0x55e90503b7d0 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e904f89ca0 con 0x55e905040580 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09ad5ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e9050409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09ad5ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e9050409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:38526/0 (socket says 172.21.15.37:38526) 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09ad5ba700 1 -- 172.21.15.37:0/855975494 learned_addr learned my addr 172.21.15.37:0/855975494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09acdb9700 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e90503bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.379+0000 7f09a7fff700 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e905040040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 -- 172.21.15.37:0/855975494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 msgr2=0x55e9050409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e9050409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 -- 172.21.15.37:0/855975494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 msgr2=0x55e905040040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e905040040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 -- 172.21.15.37:0/855975494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e904edb880 con 0x55e90503b7d0 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f09acdb9700 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e90503bbb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f099c0cee30 tx=0x7f099c09d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81a1f75f5ac479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f0992ffd700 1 -- 172.21.15.37:0/855975494 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c09dd40 con 0x55e90503b7d0 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f0992ffd700 1 -- 172.21.15.37:0/855975494 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c09d640 con 0x55e90503b7d0 2024-04-25T21:08:57.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.380+0000 7f0992ffd700 1 -- 172.21.15.37:0/855975494 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c09db00 con 0x55e90503b7d0 2024-04-25T21:08:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- 172.21.15.37:0/855975494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 msgr2=0x55e90503bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e90503bbb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f099c0cee30 tx=0x7f099c09d490 comp rx=0 tx=0).stop 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- 172.21.15.37:0/855975494 shutdown_connections 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e9050409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e90503bbb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/855975494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e905040040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- 172.21.15.37:0/855975494 >> 172.21.15.37:0/855975494 conn(0x55e904f68640 msgr2=0x55e905036820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- 172.21.15.37:0/855975494 shutdown_connections 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- 172.21.15.37:0/855975494 wait complete. 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 -- start start 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.381+0000 7f09b98a6b80 1 Processor -- start 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905044d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e905040580 0x55e9050452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e904f34910 con 0x55e90503b7d0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e904f2cb80 con 0x55e905040580 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e905048460 con 0x55e90503c0f0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09b98a6b80 10 client.0 _ll_register_callbacks cb 0x55e904f26900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09acdb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09acdb9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50848/0 (socket says 172.21.15.37:50848) 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09acdb9700 1 -- 172.21.15.37:0/3009249084 learned_addr learned my addr 172.21.15.37:0/3009249084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09a7fff700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905044d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.382+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e905040580 0x55e9050452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 -- 172.21.15.37:0/3009249084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 msgr2=0x55e905044d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905044d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 -- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e905040580 msgr2=0x55e9050452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e905040580 0x55e9050452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 -- 172.21.15.37:0/3009249084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e904f45220 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09acdb9700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c010 tx=0x7f099c0cee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c750a530e1daee74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f0987fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0bb060 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f099c0a5fa0 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c09d640 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f099c0b6030 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09b98a6b80 10 client.15375 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e905052ae0 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.383+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c09d640 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f099c0a2b80 con 0x55e90503b7d0 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 10 client.15375 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 20 client.15375 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e905054c90 0x55e905056100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 4 client.15375 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:08:57.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e904f9b4d0 con 0x55e905054c90 2024-04-25T21:08:57.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.384+0000 7f09a7fff700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e905054c90 0x55e905056100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.385+0000 7f09a7fff700 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e905054c90 0x55e905056100 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57323e666a75dd2b server_cookie=31b85cda36cf1b50 in_seq=0 out_seq=0 2024-04-25T21:08:57.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.385+0000 7f0987fff700 10 client.15375 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.385+0000 7f0987fff700 1 -- 172.21.15.37:0/3009249084 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e904f9b4d0 con 0x55e905054c90 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.385+0000 7f0987fff700 10 client.15375 handle_command_reply: tid=0 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.385+0000 7f09b98a6b80 1 client.15375 shutdown 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 msgr2=0x55e905047a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c010 tx=0x7f099c0cee90 comp rx=0 tx=0).stop 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 shutdown_connections 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e905040580 0x55e9050452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e905054c90 0x55e905056100 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503b7d0 0x55e905047a90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/3009249084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905044d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 >> 172.21.15.37:0/3009249084 conn(0x55e904f68640 msgr2=0x55e9050419e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f0987fff700 0 client.15375 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 shutdown_connections 2024-04-25T21:08:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 1 -- 172.21.15.37:0/3009249084 wait complete. 2024-04-25T21:08:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.386+0000 7f09b98a6b80 20 client.15375 trim_cache size 0 max 16384 2024-04-25T21:08:57.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 Processor -- start 2024-04-25T21:08:57.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 -- start start 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905042940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e90503ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e905054440 con 0x55e90503c0f0 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e905054b10 con 0x55e90503b7d0 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e9050551e0 con 0x55e905040580 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50850/0 (socket says 172.21.15.37:50850) 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09a7fff700 1 -- 172.21.15.37:0/660010087 learned_addr learned my addr 172.21.15.37:0/660010087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09acdb9700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905042940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e90503ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.410+0000 7f09acdb9700 1 -- 172.21.15.37:0/660010087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 msgr2=0x55e90503ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09acdb9700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e90503ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09acdb9700 1 -- 172.21.15.37:0/660010087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 msgr2=0x55e90503a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09acdb9700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09acdb9700 1 -- 172.21.15.37:0/660010087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e905035870 con 0x55e90503b7d0 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09a7fff700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e90503ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09acdb9700 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905042940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f099c029000 tx=0x7f099c09d060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fd2165914650c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09927fc700 1 -- 172.21.15.37:0/660010087 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0bb070 con 0x55e90503b7d0 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09927fc700 1 -- 172.21.15.37:0/660010087 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c003680 con 0x55e90503b7d0 2024-04-25T21:08:57.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.411+0000 7f09927fc700 1 -- 172.21.15.37:0/660010087 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c09d640 con 0x55e90503b7d0 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- 172.21.15.37:0/660010087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 msgr2=0x55e905042940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905042940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f099c029000 tx=0x7f099c09d060 comp rx=0 tx=0).stop 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- 172.21.15.37:0/660010087 shutdown_connections 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e905040580 0x55e90503ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905042940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/660010087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e90503c0f0 0x55e90503a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- 172.21.15.37:0/660010087 >> 172.21.15.37:0/660010087 conn(0x55e904f68640 msgr2=0x55e905042230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- 172.21.15.37:0/660010087 shutdown_connections 2024-04-25T21:08:57.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- 172.21.15.37:0/660010087 wait complete. 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 -- start start 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.412+0000 7f09b98a6b80 1 Processor -- start 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905040b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905041080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e905055f80 con 0x55e905040580 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e9050558b0 con 0x55e90503b7d0 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e905056650 con 0x55e90503c0f0 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09b98a6b80 10 client.0 _ll_register_callbacks cb 0x55e904f26900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09ad5ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09ad5ba700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50866/0 (socket says 172.21.15.37:50866) 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09ad5ba700 1 -- 172.21.15.37:0/1993715807 learned_addr learned my addr 172.21.15.37:0/1993715807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:08:57.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09acdb9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905040b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.413+0000 7f09a7fff700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905041080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 -- 172.21.15.37:0/1993715807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 msgr2=0x55e905041080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905041080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 -- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 msgr2=0x55e905040b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905040b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 -- 172.21.15.37:0/1993715807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90505eb30 con 0x55e905040580 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09ad5ba700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09a0007b20 tx=0x7f09a0007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3b815466640cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f0987fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09acdb9700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905040b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a00188f0 con 0x55e905040580 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c003680 con 0x55e905040580 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f09a0018a90 con 0x55e905040580 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09a0018c90 con 0x55e905040580 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09b98a6b80 10 client.15381 fetch_fsmap learned FSMap version 27 2024-04-25T21:08:57.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.414+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9050bbdc0 con 0x55e905040580 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c003680 con 0x55e905040580 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f09a001f070 con 0x55e905040580 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 10 client.15381 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 20 client.15381 populate_metadata read hostname 'smithi037' 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e9050c08c0 0x55e9050bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 4 client.15381 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 --> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] -- command(tid 0: {"prefix": "status"}) -- 0x55e904f95570 con 0x55e9050c08c0 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.415+0000 7f09acdb9700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e9050c08c0 0x55e9050bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f0987fff700 10 client.15381 ms_handle_connect on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09acdb9700 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e9050c08c0 0x55e9050bf5d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0c794067a738406 server_cookie=e9c877f9295adaea in_seq=0 out_seq=0 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f0987fff700 1 -- 172.21.15.37:0/1993715807 <== mds.4 v2:172.21.15.98:6836/2446670971 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e904f95570 con 0x55e9050c08c0 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f0987fff700 10 client.15381 handle_command_reply: tid=0 2024-04-25T21:08:57.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 client.15381 shutdown 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 msgr2=0x55e90505e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f09a0007b20 tx=0x7f09a0007b50 comp rx=0 tx=0).stop 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 shutdown_connections 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:6836/2446670971,v1:172.21.15.98:6837/2446670971] conn(0x55e9050c08c0 0x55e9050bf5d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e905040580 0x55e90505e570 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e90503b7d0 0x55e905040b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f0987fff700 0 client.15381 ms_handle_reset on v2:172.21.15.98:6836/2446670971 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 --2- 172.21.15.37:0/1993715807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e90503c0f0 0x55e905041080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.416+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 >> 172.21.15.37:0/1993715807 conn(0x55e904f68640 msgr2=0x55e90503fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 shutdown_connections 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09b98a6b80 1 -- 172.21.15.37:0/1993715807 wait complete. 2024-04-25T21:08:57.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09b98a6b80 20 client.15381 trim_cache size 0 max 16384 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 4, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24419, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 217.65137166100001, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 264.36221929999999 2024-04-25T21:08:57.422 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:08:57.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 msgr2=0x7f09980a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 0x7f09980a6410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f09a8104560 tx=0x7f099400b3c0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 msgr2=0x7f09a8110a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.417+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c0057c0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 shutdown_connections 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f09980a3f50 0x7f09980a6410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09a8106cc0 0x7f09a8111510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a81023a0 0x7f09a8110a90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 --2- 172.21.15.37:0/2116397916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09a8102d50 0x7f09a8110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 >> 172.21.15.37:0/2116397916 conn(0x7f09a8077fb0 msgr2=0x7f09a80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 shutdown_connections 2024-04-25T21:08:57.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:08:57.418+0000 7f09af01d700 1 -- 172.21.15.37:0/2116397916 wait complete. 2024-04-25T21:08:57.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:57 smithi098 ceph-mon[32569]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T21:08:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:57 smithi037 ceph-mon[29195]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T21:08:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:57 smithi177 ceph-mon[33043]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.4 KiB/s wr, 15 op/s 2024-04-25T21:08:58.248 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.f status 2024-04-25T21:08:58.526 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:59.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:08:59 smithi037 ceph-mon[29195]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T21:09:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:08:59 smithi177 ceph-mon[33043]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T21:09:00.170 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:08:59 smithi098 ceph-mon[32569]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-25T21:09:00.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- 172.21.15.37:0/3398292360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c107990 msgr2=0x7f197c107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 --2- 172.21.15.37:0/3398292360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c107990 0x7f197c107d70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f196c009990 tx=0x7f196c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- 172.21.15.37:0/3398292360 shutdown_connections 2024-04-25T21:09:00.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 --2- 172.21.15.37:0/3398292360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c102300 0x7f197c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 --2- 172.21.15.37:0/3398292360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c107990 0x7f197c107d70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 --2- 172.21.15.37:0/3398292360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c101940 0x7f197c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- 172.21.15.37:0/3398292360 >> 172.21.15.37:0/3398292360 conn(0x7f197c0788f0 msgr2=0x7f197c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- 172.21.15.37:0/3398292360 shutdown_connections 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- 172.21.15.37:0/3398292360 wait complete. 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 Processor -- start 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.368+0000 7f19828fc700 1 -- start start 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c101940 0x7f197c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c107990 0x7f197c1a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111ae0 con 0x7f197c102300 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111960 con 0x7f197c101940 2024-04-25T21:09:00.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f19828fc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f197c111c60 con 0x7f197c107990 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39482/0 (socket says 172.21.15.37:39482) 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 -- 172.21.15.37:0/122595369 learned_addr learned my addr 172.21.15.37:0/122595369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f1980e99700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c107990 0x7f197c1a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197bfff700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c101940 0x7f197c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 -- 172.21.15.37:0/122595369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c107990 msgr2=0x7f197c1a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c107990 0x7f197c1a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 -- 172.21.15.37:0/122595369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c101940 msgr2=0x7f197c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c101940 0x7f197c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 -- 172.21.15.37:0/122595369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c1a5480 con 0x7f197c102300 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.369+0000 7f197b7fe700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f197000d950 tx=0x7f197000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.370+0000 7f19797fa700 1 -- 172.21.15.37:0/122595369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970016070 con 0x7f197c102300 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.370+0000 7f19797fa700 1 -- 172.21.15.37:0/122595369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197000fd40 con 0x7f197c102300 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.370+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c1a5710 con 0x7f197c102300 2024-04-25T21:09:00.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.370+0000 7f19797fa700 1 -- 172.21.15.37:0/122595369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197000e3b0 con 0x7f197c102300 2024-04-25T21:09:00.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.370+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c1a5c20 con 0x7f197c102300 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.372+0000 7f19797fa700 1 -- 172.21.15.37:0/122595369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f1970019070 con 0x7f197c102300 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.373+0000 7f19797fa700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 0x7f19640a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.373+0000 7f197bfff700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 0x7f19640a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.373+0000 7f19797fa700 1 -- 172.21.15.37:0/122595369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f19700d08c0 con 0x7f197c102300 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.373+0000 7f198d185b80 1 Processor -- start 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 -- start start 2024-04-25T21:09:00.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197bfff700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 0x7f19640a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098e0 tx=0x7f196c005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e35490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e42c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562047d88c40 con 0x562047e359d0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562047d2f8e0 con 0x562047e350b0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f198d185b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562047d84c00 con 0x562047e40880 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39496/0 (socket says 172.21.15.37:39496) 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 -- 172.21.15.37:0/1866719922 learned_addr learned my addr 172.21.15.37:0/1866719922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197bfff700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e35490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f1980e99700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e42c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 -- 172.21.15.37:0/1866719922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 msgr2=0x562047e42c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e42c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 -- 172.21.15.37:0/1866719922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 msgr2=0x562047e35490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e35490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197b7fe700 1 -- 172.21.15.37:0/1866719922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562047cf36f0 con 0x562047e359d0 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.374+0000 7f197bfff700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.375+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19700ce640 tx=0x7f197009ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5da761ebb9a77a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.375+0000 7f19627fc700 1 -- 172.21.15.37:0/1866719922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970020070 con 0x562047e359d0 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.375+0000 7f19627fc700 1 -- 172.21.15.37:0/1866719922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19700a09f0 con 0x562047e359d0 2024-04-25T21:09:00.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.375+0000 7f19627fc700 1 -- 172.21.15.37:0/1866719922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197009cb00 con 0x562047e359d0 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- 172.21.15.37:0/1866719922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 msgr2=0x562047e40340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f19700ce640 tx=0x7f197009ce10 comp rx=0 tx=0).stop 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- 172.21.15.37:0/1866719922 shutdown_connections 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e42c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e35490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 --2- 172.21.15.37:0/1866719922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e40340 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- 172.21.15.37:0/1866719922 >> 172.21.15.37:0/1866719922 conn(0x562047d62840 msgr2=0x562047e31580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- 172.21.15.37:0/1866719922 shutdown_connections 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- 172.21.15.37:0/1866719922 wait complete. 2024-04-25T21:09:00.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 -- start start 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.376+0000 7f198d185b80 1 Processor -- start 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e40030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39512/0 (socket says 172.21.15.37:39512) 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e3d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 -- 172.21.15.37:0/1142939731 learned_addr learned my addr 172.21.15.37:0/1142939731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562047e43250 con 0x562047e359d0 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562047d42560 con 0x562047e350b0 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562047e433d0 con 0x562047e40880 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197bfff700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e40030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f198d185b80 10 client.0 _ll_register_callbacks cb 0x562047d218d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:00.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f1980e99700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e3d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 -- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 msgr2=0x562047e3d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e3d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 -- 172.21.15.37:0/1142939731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 msgr2=0x562047e40030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e40030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197b7fe700 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562047cd6880 con 0x562047e359d0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f1980e99700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e3d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.377+0000 7f197bfff700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e40030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f197b7fe700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1970000c00 tx=0x7f19700cdf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a655783dd142479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970020070 con 0x562047e359d0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f197009c8a0 con 0x562047e359d0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19700a09f0 con 0x562047e359d0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f197009caa0 con 0x562047e359d0 2024-04-25T21:09:00.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f198d185b80 10 client.15393 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562047e4dad0 con 0x562047e359d0 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19700a09f0 con 0x562047e359d0 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1970085b80 con 0x562047e359d0 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f198d185b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.378+0000 7f198d185b80 10 client.15393 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f198d185b80 20 client.15393 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047e51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f198d185b80 4 client.15393 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562047d96430 con 0x562047e4fc00 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f197bfff700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047e51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f197bfff700 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047e51070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=162b62a69c06936 server_cookie=e0858cf93dc1e36a in_seq=0 out_seq=0 2024-04-25T21:09:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.379+0000 7f195b7fe700 10 client.15393 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f195b7fe700 1 -- 172.21.15.37:0/1142939731 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562047d96430 con 0x562047e4fc00 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f195b7fe700 10 client.15393 handle_command_reply: tid=0 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 client.15393 shutdown 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 msgr2=0x562047e3d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1970000c00 tx=0x7f19700cdf40 comp rx=0 tx=0).stop 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 shutdown_connections 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047e51070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047e3d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f195b7fe700 0 client.15393 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.380+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e350b0 0x562047e40030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.381+0000 7f198d185b80 1 --2- 172.21.15.37:0/1142939731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e359d0 0x562047e3d330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.381+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 >> 172.21.15.37:0/1142939731 conn(0x562047d62840 msgr2=0x562047e311b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.381+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 shutdown_connections 2024-04-25T21:09:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.381+0000 7f198d185b80 1 -- 172.21.15.37:0/1142939731 wait complete. 2024-04-25T21:09:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.381+0000 7f198d185b80 20 client.15393 trim_cache size 0 max 16384 2024-04-25T21:09:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.406+0000 7f198d185b80 1 Processor -- start 2024-04-25T21:09:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.406+0000 7f198d185b80 1 -- start start 2024-04-25T21:09:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e350b0 0x562047e3c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e5a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562047e4f3b0 con 0x562047e40880 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562047e4fa80 con 0x562047e359d0 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f198d185b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562047e50150 con 0x562047e350b0 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39522/0 (socket says 172.21.15.37:39522) 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 -- 172.21.15.37:0/2487060247 learned_addr learned my addr 172.21.15.37:0/2487060247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f197b7fe700 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e5a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f197bfff700 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e350b0 0x562047e3c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 -- 172.21.15.37:0/2487060247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e350b0 msgr2=0x562047e3c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e350b0 0x562047e3c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 -- 172.21.15.37:0/2487060247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 msgr2=0x562047e5a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e5a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 -- 172.21.15.37:0/2487060247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562047e35df0 con 0x562047e40880 2024-04-25T21:09:00.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.407+0000 7f1980e99700 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f19740079e0 tx=0x7f1974009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8520da1c4c652e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f1961ffb700 1 -- 172.21.15.37:0/2487060247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974003e10 con 0x562047e40880 2024-04-25T21:09:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f1961ffb700 1 -- 172.21.15.37:0/2487060247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974004500 con 0x562047e40880 2024-04-25T21:09:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f1961ffb700 1 -- 172.21.15.37:0/2487060247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974017af0 con 0x562047e40880 2024-04-25T21:09:00.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 -- 172.21.15.37:0/2487060247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 msgr2=0x562047e5aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f19740079e0 tx=0x7f1974009920 comp rx=0 tx=0).stop 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 -- 172.21.15.37:0/2487060247 shutdown_connections 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e40880 0x562047e5aa20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e350b0 0x562047e3c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 --2- 172.21.15.37:0/2487060247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e5a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 -- 172.21.15.37:0/2487060247 >> 172.21.15.37:0/2487060247 conn(0x562047d62840 msgr2=0x562047e3cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 -- 172.21.15.37:0/2487060247 shutdown_connections 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.408+0000 7f198d185b80 1 -- 172.21.15.37:0/2487060247 wait complete. 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 -- start start 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 Processor -- start 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e56ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047eb5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562047e50ef0 con 0x562047e350b0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562047e50820 con 0x562047e359d0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562047e515c0 con 0x562047e40880 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f198d185b80 10 client.0 _ll_register_callbacks cb 0x562047d218d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f197bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f197bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39526/0 (socket says 172.21.15.37:39526) 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f197bfff700 1 -- 172.21.15.37:0/446786869 learned_addr learned my addr 172.21.15.37:0/446786869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f1980e99700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047eb5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.409+0000 7f197b7fe700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e56ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 -- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 msgr2=0x562047eb5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047eb5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 -- 172.21.15.37:0/446786869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 msgr2=0x562047e56ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e56ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 -- 172.21.15.37:0/446786869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562047eb6270 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f197bfff700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f196c009990 tx=0x7f196c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8feee99cde45a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f195b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c038060 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f196c003a90 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1974004500 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f196c003c90 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f198d185b80 10 client.15399 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562047eb75a0 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.410+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974004500 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f196c03d480 con 0x562047e350b0 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 10 client.15399 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 20 client.15399 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047ebae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 4 client.15399 mds_command: new command op to 24445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 --> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] -- command(tid 0: {"prefix": "status"}) -- 0x562047d904d0 con 0x562047e4fc00 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f197b7fe700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047ebae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f195b7fe700 10 client.15399 ms_handle_connect on v2:172.21.15.177:6834/724905952 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f197b7fe700 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047ebae90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fe0243f49571da82 server_cookie=2f835998f2d1e543 in_seq=0 out_seq=0 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.411+0000 7f195b7fe700 1 -- 172.21.15.37:0/446786869 <== mds.3 v2:172.21.15.177:6834/724905952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562047d904d0 con 0x562047e4fc00 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f195b7fe700 10 client.15399 handle_command_reply: tid=0 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 client.15399 shutdown 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 msgr2=0x562047e56590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f196c009990 tx=0x7f196c01f000 comp rx=0 tx=0).stop 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 shutdown_connections 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:6834/724905952,v1:172.21.15.177:6835/724905952] conn(0x562047e4fc00 0x562047ebae90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562047e40880 0x562047eb5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562047e350b0 0x562047e56590 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f195b7fe700 0 client.15399 ms_handle_reset on v2:172.21.15.177:6834/724905952 2024-04-25T21:09:00.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 --2- 172.21.15.37:0/446786869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562047e359d0 0x562047e56ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 >> 172.21.15.37:0/446786869 conn(0x562047d62840 msgr2=0x562047e3d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 shutdown_connections 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 1 -- 172.21.15.37:0/446786869 wait complete. 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f198d185b80 20 client.15399 trim_cache size 0 max 16384 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 3, 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24445, 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:09:00.417 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 221.65250809400001, 2024-04-25T21:09:00.418 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T21:09:00.418 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T21:09:00.418 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T21:09:00.418 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 273.18415526299998 2024-04-25T21:09:00.418 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 msgr2=0x7f19640a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 0x7f19640a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f196c0098e0 tx=0x7f196c005960 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 msgr2=0x7f197c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.412+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f197000d950 tx=0x7f197000dc60 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 shutdown_connections 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19640a3ed0 0x7f19640a6390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f197c107990 0x7f197c1a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f197c101940 0x7f197c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 --2- 172.21.15.37:0/122595369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f197c102300 0x7f197c19a400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 >> 172.21.15.37:0/122595369 conn(0x7f197c0788f0 msgr2=0x7f197c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 shutdown_connections 2024-04-25T21:09:00.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:00.413+0000 7f19828fc700 1 -- 172.21.15.37:0/122595369 wait complete. 2024-04-25T21:09:01.383 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:09:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:09:00.986+0000 7f675120b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:09:01.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:09:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:09:00.986+0000 7f675120b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:09:01.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:09:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:09:00.986+0000 7f675120b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:09:01.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:09:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:09:00.986+0000 7f675120b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:09:01.384 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:09:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:09:00.986+0000 7f675120b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T21:09:01.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-25T21:09:01.664 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:09:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:01 smithi037 ceph-mon[29195]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-04-25T21:09:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:01 smithi177 ceph-mon[33043]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-04-25T21:09:02.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:01 smithi098 ceph-mon[32569]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.9 KiB/s wr, 15 op/s 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2287058775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c06b980 msgr2=0x7f2b2c10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2287058775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c06b980 0x7f2b2c10f8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b0a0 tx=0x7f2b2400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2287058775 shutdown_connections 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2287058775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c06b980 0x7f2b2c10f8f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2287058775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c108a20 0x7f2b2c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2287058775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2287058775 >> 172.21.15.37:0/2287058775 conn(0x7f2b2c0788b0 msgr2=0x7f2b2c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2287058775 shutdown_connections 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.538+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2287058775 wait complete. 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 Processor -- start 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 -- start start 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06b980 0x7f2b2c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 0x7f2b2c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c112ba0 con 0x7f2b2c108a20 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c112a20 con 0x7f2b2c06b980 2024-04-25T21:09:03.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b32bd4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c112d20 con 0x7f2b2c06afe0 2024-04-25T21:09:03.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b30970700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b30970700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:35050/0 (socket says 172.21.15.37:35050) 2024-04-25T21:09:03.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b30970700 1 -- 172.21.15.37:0/2686919651 learned_addr learned my addr 172.21.15.37:0/2686919651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b31171700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 0x7f2b2c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.539+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06b980 0x7f2b2c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 -- 172.21.15.37:0/2686919651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 msgr2=0x7f2b2c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 -- 172.21.15.37:0/2686919651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06b980 msgr2=0x7f2b2c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06b980 0x7f2b2c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 -- 172.21.15.37:0/2686919651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c19e090 con 0x7f2b2c108a20 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b31171700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 0x7f2b2c19d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b070 tx=0x7f2b24004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b29ffb700 1 -- 172.21.15.37:0/2686919651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24021070 con 0x7f2b2c108a20 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c19e320 con 0x7f2b2c108a20 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b29ffb700 1 -- 172.21.15.37:0/2686919651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b24007b80 con 0x7f2b2c108a20 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b29ffb700 1 -- 172.21.15.37:0/2686919651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b24005040 con 0x7f2b2c108a20 2024-04-25T21:09:03.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.540+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c1a5c00 con 0x7f2b2c108a20 2024-04-25T21:09:03.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.543+0000 7f2b29ffb700 1 -- 172.21.15.37:0/2686919651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f2b24024070 con 0x7f2b2c108a20 2024-04-25T21:09:03.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.544+0000 7f2b29ffb700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 0x7f2b1c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.544+0000 7f2b30970700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 0x7f2b1c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.544+0000 7f2b29ffb700 1 -- 172.21.15.37:0/2686919651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2b2400f020 con 0x7f2b2c108a20 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b30970700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 0x7f2b1c0a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005950 tx=0x7f2b2000b500 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 Processor -- start 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 -- start start 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 0x5640f82dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640f8228ce0 con 0x5640f82db500 2024-04-25T21:09:03.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640f81cf910 con 0x5640f82dabe0 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5640f8224ca0 con 0x5640f82df990 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39558/0 (socket says 172.21.15.37:39558) 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 -- 172.21.15.37:0/474067542 learned_addr learned my addr 172.21.15.37:0/474067542 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b30970700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 0x5640f82dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b31171700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 -- 172.21.15.37:0/474067542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 msgr2=0x5640f82dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82dfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 -- 172.21.15.37:0/474067542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 msgr2=0x5640f82dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 0x5640f82dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b2bfff700 1 -- 172.21.15.37:0/474067542 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f8176880 con 0x5640f82db500 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.545+0000 7f2b30970700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 0x5640f82dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007c90 tx=0x7f2b180103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3519d262101fb018 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b16ffd700 1 -- 172.21.15.37:0/474067542 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18010d00 con 0x5640f82db500 2024-04-25T21:09:03.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b16ffd700 1 -- 172.21.15.37:0/474067542 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b18015070 con 0x5640f82db500 2024-04-25T21:09:03.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b16ffd700 1 -- 172.21.15.37:0/474067542 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b18003dd0 con 0x5640f82db500 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b3d45db80 1 -- 172.21.15.37:0/474067542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 msgr2=0x5640f82df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.546+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2b18007c90 tx=0x7f2b180103d0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 -- 172.21.15.37:0/474067542 shutdown_connections 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82dabe0 0x5640f82dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/474067542 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82db500 0x5640f82df450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 -- 172.21.15.37:0/474067542 >> 172.21.15.37:0/474067542 conn(0x5640f8203640 msgr2=0x5640f82d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 -- 172.21.15.37:0/474067542 shutdown_connections 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 -- 172.21.15.37:0/474067542 wait complete. 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 -- start start 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.547+0000 7f2b3d45db80 1 Processor -- start 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82db500 0x5640f82e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82df990 0x5640f82e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640f82e7a20 con 0x5640f82dabe0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640f81c7b80 con 0x5640f82df990 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5640f82e7ba0 con 0x5640f82db500 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 10 client.0 _ll_register_callbacks cb 0x5640f81c1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39566/0 (socket says 172.21.15.37:39566) 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 -- 172.21.15.37:0/2751969194 learned_addr learned my addr 172.21.15.37:0/2751969194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b31171700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82df990 0x5640f82e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82db500 0x5640f82e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 -- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82db500 msgr2=0x5640f82e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82db500 0x5640f82e41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 -- 172.21.15.37:0/2751969194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82df990 msgr2=0x5640f82e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82df990 0x5640f82e46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 -- 172.21.15.37:0/2751969194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f81e0220 con 0x5640f82dabe0 2024-04-25T21:09:03.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b30970700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b20011740 tx=0x7f2b200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150096bc5f4e5f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004460 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2b20003680 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.548+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b18015070 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b200038c0 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 10 client.15417 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5640f82f2330 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b18015070 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2b200319b0 con 0x5640f82dabe0 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 10 client.15417 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 10 client.15417 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 20 client.15417 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f82f4990 0x5640f82f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 4 client.15417 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.549+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f82364d0 con 0x5640f82f4990 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.550+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f82f4990 0x5640f82f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.550+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f82f4990 0x5640f82f5e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9226a4ec05c44fe server_cookie=bd3d190af77cad7b in_seq=0 out_seq=0 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.550+0000 7f2b0bfff700 10 client.15417 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b0bfff700 1 -- 172.21.15.37:0/2751969194 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640f82364d0 con 0x5640f82f4990 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b0bfff700 10 client.15417 handle_command_reply: tid=0 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 client.15417 shutdown 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 msgr2=0x5640f82e6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b20011740 tx=0x7f2b200047c0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 shutdown_connections 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82df990 0x5640f82e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e6eb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f82f4990 0x5640f82f5e00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/2751969194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82db500 0x5640f82e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b0bfff700 0 client.15417 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 >> 172.21.15.37:0/2751969194 conn(0x5640f8203640 msgr2=0x5640f82e0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 shutdown_connections 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 1 -- 172.21.15.37:0/2751969194 wait complete. 2024-04-25T21:09:03.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.551+0000 7f2b3d45db80 20 client.15417 trim_cache size 0 max 16384 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.572+0000 7f2b3d45db80 1 Processor -- start 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.572+0000 7f2b3d45db80 1 -- start start 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f4140 con 0x5640f82dabe0 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f4810 con 0x5640f82db500 2024-04-25T21:09:03.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f4ee0 con 0x5640f82df990 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39580/0 (socket says 172.21.15.37:39580) 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 -- 172.21.15.37:0/4181414128 learned_addr learned my addr 172.21.15.37:0/4181414128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b31171700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 -- 172.21.15.37:0/4181414128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 msgr2=0x5640f82e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 -- 172.21.15.37:0/4181414128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 msgr2=0x5640f82e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 -- 172.21.15.37:0/4181414128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f82df7a0 con 0x5640f82dabe0 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.573+0000 7f2b30970700 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b20010f90 tx=0x7f2b20010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff352e76fc2a2b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.574+0000 7f2b167fc700 1 -- 172.21.15.37:0/4181414128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20040070 con 0x5640f82dabe0 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.574+0000 7f2b167fc700 1 -- 172.21.15.37:0/4181414128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b20023030 con 0x5640f82dabe0 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.574+0000 7f2b167fc700 1 -- 172.21.15.37:0/4181414128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037040 con 0x5640f82dabe0 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4181414128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 msgr2=0x5640f82e4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b20010f90 tx=0x7f2b20010fc0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4181414128 shutdown_connections 2024-04-25T21:09:03.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82dabe0 0x5640f82e4720 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4181414128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4181414128 >> 172.21.15.37:0/4181414128 conn(0x5640f8203640 msgr2=0x5640f82d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4181414128 shutdown_connections 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4181414128 wait complete. 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.575+0000 7f2b3d45db80 1 -- start start 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 Processor -- start 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82e0460 0x5640f82e08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f5c80 con 0x5640f82e0460 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f55b0 con 0x5640f82db500 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5640f82f6350 con 0x5640f82df990 2024-04-25T21:09:03.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b3d45db80 10 client.0 _ll_register_callbacks cb 0x5640f81c1900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b30970700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b2bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:35074/0 (socket says 172.21.15.37:35074) 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b2bfff700 1 -- 172.21.15.37:0/4137124551 learned_addr learned my addr 172.21.15.37:0/4137124551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.576+0000 7f2b31171700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82e0460 0x5640f82e08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 -- 172.21.15.37:0/4137124551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 msgr2=0x5640f82ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 -- 172.21.15.37:0/4137124551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82e0460 msgr2=0x5640f82e08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82e0460 0x5640f82e08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 -- 172.21.15.37:0/4137124551 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f82e0f20 con 0x5640f82df990 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b2bfff700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b18003a20 tx=0x7f2b18003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6d78dcc681dcf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1802f060 con 0x5640f82df990 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b20023030 con 0x5640f82df990 2024-04-25T21:09:03.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.577+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2b18003c80 con 0x5640f82df990 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2b18010e50 con 0x5640f82df990 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 10 client.25060 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5640f835c8a0 con 0x5640f82df990 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b20023030 con 0x5640f82df990 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2b180294b0 con 0x5640f82df990 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 10 client.25060 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 10 client.25060 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 20 client.25060 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f8361270 0x5640f835ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 4 client.25060 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.578+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 --> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] -- command(tid 0: {"prefix": "status"}) -- 0x5640f8230570 con 0x5640f8361270 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b30970700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f8361270 0x5640f835ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:03.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b0bfff700 10 client.25060 ms_handle_connect on v2:172.21.15.177:6836/2828024557 2024-04-25T21:09:03.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b30970700 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f8361270 0x5640f835ff80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97eea31e887629b4 server_cookie=b2a1689a3b321095 in_seq=0 out_seq=0 2024-04-25T21:09:03.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b0bfff700 1 -- 172.21.15.37:0/4137124551 <== mds.2 v2:172.21.15.177:6836/2828024557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5640f8230570 con 0x5640f8361270 2024-04-25T21:09:03.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b0bfff700 10 client.25060 handle_command_reply: tid=0 2024-04-25T21:09:03.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b3d45db80 1 client.25060 shutdown 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 msgr2=0x5640f82ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.579+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b18003a20 tx=0x7f2b18003710 comp rx=0 tx=0).stop 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 shutdown_connections 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640f82e0460 0x5640f82e08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5640f82db500 0x5640f82ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:6836/2828024557,v1:172.21.15.177:6837/2828024557] conn(0x5640f8361270 0x5640f835ff80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 --2- 172.21.15.37:0/4137124551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5640f82df990 0x5640f82ff7b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 >> 172.21.15.37:0/4137124551 conn(0x5640f8203640 msgr2=0x5640f82e2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b0bfff700 0 client.25060 ms_handle_reset on v2:172.21.15.177:6836/2828024557 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 shutdown_connections 2024-04-25T21:09:03.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 1 -- 172.21.15.37:0/4137124551 wait complete. 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.580+0000 7f2b3d45db80 20 client.25060 trim_cache size 0 max 16384 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24457, 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 225.83517529100001, 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T21:09:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T21:09:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T21:09:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 273.58705777500001 2024-04-25T21:09:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:09:03.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 msgr2=0x7f2b1c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 0x7f2b1c0a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b20005950 tx=0x7f2b2000b500 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 msgr2=0x7f2b2c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 0x7f2b2c19d950 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b070 tx=0x7f2b24004ce0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 shutdown_connections 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2b1c0a3f60 0x7f2b1c0a6420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b2c108a20 0x7f2b2c19d950 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b2c06afe0 0x7f2b2c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 --2- 172.21.15.37:0/2686919651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b2c06b980 0x7f2b2c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 >> 172.21.15.37:0/2686919651 conn(0x7f2b2c0788b0 msgr2=0x7f2b2c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 shutdown_connections 2024-04-25T21:09:03.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:03.581+0000 7f2b32bd4700 1 -- 172.21.15.37:0/2686919651 wait complete. 2024-04-25T21:09:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:03 smithi037 ceph-mon[29195]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-25T21:09:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:03 smithi037 ceph-mon[29195]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-04-25T21:09:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:03 smithi177 ceph-mon[33043]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-25T21:09:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:03 smithi177 ceph-mon[33043]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-04-25T21:09:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:03 smithi098 ceph-mon[32569]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-25T21:09:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:03 smithi098 ceph-mon[32569]: mgrmap e25: x(active, since 34s), standbys: y, z 2024-04-25T21:09:04.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T21:09:04.596 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:09:05.966 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:05 smithi037 ceph-mon[29195]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T21:09:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:05 smithi177 ceph-mon[33043]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T21:09:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:05 smithi098 ceph-mon[32569]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2024-04-25T21:09:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 -- 172.21.15.37:0/887223132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 msgr2=0x7f6fe010efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/887223132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe010efa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009990 tx=0x7f6fd0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 -- 172.21.15.37:0/887223132 shutdown_connections 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/887223132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe010f4e0 0x7f6fe01118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/887223132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe006afe0 0x7f6fe006b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/887223132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe010efa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 -- 172.21.15.37:0/887223132 >> 172.21.15.37:0/887223132 conn(0x7f6fe00788b0 msgr2=0x7f6fe0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 -- 172.21.15.37:0/887223132 shutdown_connections 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.367+0000 7f6fe80a3700 1 -- 172.21.15.37:0/887223132 wait complete. 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 Processor -- start 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 -- start start 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 0x7f6fe019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 0x7f6fe019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe0114e10 con 0x7f6fe006b900 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe0114c90 con 0x7f6fe006afe0 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe80a3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe0114f90 con 0x7f6fe010f4e0 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe563e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe563e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39610/0 (socket says 172.21.15.37:39610) 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.368+0000 7f6fe563e700 1 -- 172.21.15.37:0/582200442 learned_addr learned my addr 172.21.15.37:0/582200442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe6640700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 0x7f6fe019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 0x7f6fe019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 -- 172.21.15.37:0/582200442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 msgr2=0x7f6fe019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 0x7f6fe019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 -- 172.21.15.37:0/582200442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 msgr2=0x7f6fe019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 0x7f6fe019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 -- 172.21.15.37:0/582200442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe01a03b0 con 0x7f6fe006b900 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe6640700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 0x7f6fe019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 0x7f6fe019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe563e700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00057c0 tx=0x7f6fd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fd6ffd700 1 -- 172.21.15.37:0/582200442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0027070 con 0x7f6fe006b900 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fd6ffd700 1 -- 172.21.15.37:0/582200442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd0004020 con 0x7f6fe006b900 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fd6ffd700 1 -- 172.21.15.37:0/582200442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0022bf0 con 0x7f6fe006b900 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe01a0640 con 0x7f6fe006b900 2024-04-25T21:09:06.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.369+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe01a7f20 con 0x7f6fe006b900 2024-04-25T21:09:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.371+0000 7f6fd6ffd700 1 -- 172.21.15.37:0/582200442 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6fd002a070 con 0x7f6fe006b900 2024-04-25T21:09:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.372+0000 7f6fd6ffd700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 0x7f6fcc0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.373+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 0x7f6fcc0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.373+0000 7f6fd6ffd700 1 -- 172.21.15.37:0/582200442 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6fd00d1a20 con 0x7f6fe006b900 2024-04-25T21:09:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.373+0000 7f6ff292cb80 1 Processor -- start 2024-04-25T21:09:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.373+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 0x7f6fcc0a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc005fd0 tx=0x7f6fdc005dc0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.373+0000 7f6ff292cb80 1 -- start start 2024-04-25T21:09:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edbfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2edc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edc49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d2ed11ce0 con 0x564d2edc00f0 2024-04-25T21:09:06.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564d2ecb8910 con 0x564d2edbf7d0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564d2ed0dca0 con 0x564d2edc4580 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2edc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe5e3f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edbfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe5e3f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edbfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:46710/0 (socket says 172.21.15.37:46710) 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe5e3f700 1 -- 172.21.15.37:0/973184278 learned_addr learned my addr 172.21.15.37:0/973184278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe6640700 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edc49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 -- 172.21.15.37:0/973184278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 msgr2=0x564d2edc49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edc49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 -- 172.21.15.37:0/973184278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 msgr2=0x564d2edbfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edbfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 -- 172.21.15.37:0/973184278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2ec5f880 con 0x564d2edc00f0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe6640700 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.374+0000 7f6fe563e700 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2edc4040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00cef60 tx=0x7f6fd0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9701a3697d1a9f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.375+0000 7f6fcbfff700 1 -- 172.21.15.37:0/973184278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd009de60 con 0x564d2edc00f0 2024-04-25T21:09:06.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.375+0000 7f6fcbfff700 1 -- 172.21.15.37:0/973184278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd009d760 con 0x564d2edc00f0 2024-04-25T21:09:06.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.375+0000 7f6fcbfff700 1 -- 172.21.15.37:0/973184278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00a5eb0 con 0x564d2edc00f0 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- 172.21.15.37:0/973184278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 msgr2=0x564d2edc4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2edc4040 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00cef60 tx=0x7f6fd0005a00 comp rx=0 tx=0).stop 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- 172.21.15.37:0/973184278 shutdown_connections 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edc49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edbfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/973184278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2edc4040 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- 172.21.15.37:0/973184278 >> 172.21.15.37:0/973184278 conn(0x564d2ecec640 msgr2=0x564d2edba820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- 172.21.15.37:0/973184278 shutdown_connections 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- 172.21.15.37:0/973184278 wait complete. 2024-04-25T21:09:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 -- start start 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.376+0000 7f6ff292cb80 1 Processor -- start 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edcbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc00f0 0x564d2edc8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6fe6640700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6fe6640700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39636/0 (socket says 172.21.15.37:39636) 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6fe5e3f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edcbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6fe6640700 1 -- 172.21.15.37:0/2957681062 learned_addr learned my addr 172.21.15.37:0/2957681062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d2ed11ce0 con 0x564d2edc4580 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564d2ecb0b80 con 0x564d2edbf7d0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6fe563e700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc00f0 0x564d2edc8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.377+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564d2edcc460 con 0x564d2edc00f0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6ff292cb80 10 client.0 _ll_register_callbacks cb 0x564d2ecaa900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 -- 172.21.15.37:0/2957681062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc00f0 msgr2=0x564d2edc8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc00f0 0x564d2edc8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 -- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 msgr2=0x564d2edcbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edcbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2ecc9220 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fe6640700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8007940 tx=0x7f6fd8007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea387bfc9bc9cee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fc97fa700 10 client.15435 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd009d760 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fc97fa700 1 -- 172.21.15.37:0/2957681062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8010670 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fc97fa700 1 -- 172.21.15.37:0/2957681062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6fd8010810 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.378+0000 7f6fc97fa700 1 -- 172.21.15.37:0/2957681062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fd8010a10 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 10 client.15435 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x564d2edd6a50 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6fc97fa700 1 -- 172.21.15.37:0/2957681062 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6fd8010bb0 con 0x564d2edc4580 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 10 client.15435 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 20 client.15435 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2edd9050 0x564d2edda4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 4 client.15435 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d2ed1f4d0 con 0x564d2edd9050 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.379+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2edd9050 0x564d2edda4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.380+0000 7f6fc97fa700 10 client.15435 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:09:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.380+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2edd9050 0x564d2edda4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7961ab9e110f3a6c server_cookie=aace33986404affa in_seq=0 out_seq=0 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.380+0000 7f6fc97fa700 1 -- 172.21.15.37:0/2957681062 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d2ed1f4d0 con 0x564d2edd9050 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.380+0000 7f6fc97fa700 10 client.15435 handle_command_reply: tid=0 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.380+0000 7f6ff292cb80 1 client.15435 shutdown 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 msgr2=0x564d2edc9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8007940 tx=0x7f6fd8007c50 comp rx=0 tx=0).stop 2024-04-25T21:09:06.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 shutdown_connections 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2edd9050 0x564d2edda4c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc4580 0x564d2edc9320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edbf7d0 0x564d2edcbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/2957681062 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc00f0 0x564d2edc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6fc97fa700 0 client.15435 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 >> 172.21.15.37:0/2957681062 conn(0x564d2ecec640 msgr2=0x564d2edb9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 shutdown_connections 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 1 -- 172.21.15.37:0/2957681062 wait complete. 2024-04-25T21:09:06.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.381+0000 7f6ff292cb80 20 client.15435 trim_cache size 0 max 16384 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 Processor -- start 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 -- start start 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 0x564d2edcad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc4580 0x564d2ede3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d2edd8800 con 0x564d2edc00f0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564d2edd8ed0 con 0x564d2edc4580 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564d2edd95a0 con 0x564d2edbf7d0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6fe563e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6fe563e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39648/0 (socket says 172.21.15.37:39648) 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6fe563e700 1 -- 172.21.15.37:0/3342606045 learned_addr learned my addr 172.21.15.37:0/3342606045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 0x564d2edcad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.402+0000 7f6fe6640700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc4580 0x564d2ede3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 -- 172.21.15.37:0/3342606045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 msgr2=0x564d2edcad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 0x564d2edcad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 -- 172.21.15.37:0/3342606045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc4580 msgr2=0x564d2ede3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc4580 0x564d2ede3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 -- 172.21.15.37:0/3342606045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2edbfe00 con 0x564d2edc00f0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 0x564d2edcad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fe563e700 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00a5e10 tx=0x7f6fd0088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7779c72f5a22a507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fcb7fe700 1 -- 172.21.15.37:0/3342606045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00b3030 con 0x564d2edc00f0 2024-04-25T21:09:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fcb7fe700 1 -- 172.21.15.37:0/3342606045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd00a1b10 con 0x564d2edc00f0 2024-04-25T21:09:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.403+0000 7f6fcb7fe700 1 -- 172.21.15.37:0/3342606045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00a5eb0 con 0x564d2edc00f0 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- 172.21.15.37:0/3342606045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 msgr2=0x564d2ede3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00a5e10 tx=0x7f6fd0088040 comp rx=0 tx=0).stop 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- 172.21.15.37:0/3342606045 shutdown_connections 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc4580 0x564d2ede3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edbf7d0 0x564d2edcad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/3342606045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edc00f0 0x564d2ede3920 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- 172.21.15.37:0/3342606045 >> 172.21.15.37:0/3342606045 conn(0x564d2ecec640 msgr2=0x564d2edca610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- 172.21.15.37:0/3342606045 shutdown_connections 2024-04-25T21:09:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- 172.21.15.37:0/3342606045 wait complete. 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 -- start start 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.404+0000 7f6ff292cb80 1 Processor -- start 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc00f0 0x564d2ede3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564d2edda340 con 0x564d2edbf7d0 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564d2edd9c70 con 0x564d2edc00f0 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564d2eddaa10 con 0x564d2edc4580 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 10 client.0 _ll_register_callbacks cb 0x564d2ecaa900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39656/0 (socket says 172.21.15.37:39656) 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 -- 172.21.15.37:0/1199258294 learned_addr learned my addr 172.21.15.37:0/1199258294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe563e700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc00f0 0x564d2ede3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 -- 172.21.15.37:0/1199258294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 msgr2=0x564d2edddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 -- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc00f0 msgr2=0x564d2ede3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc00f0 0x564d2ede3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 -- 172.21.15.37:0/1199258294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d2eddb420 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fe5e3f700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x564d2edcc550 tx=0x7f6fdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d5780a4a2428ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fc97fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc029800 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6fdc01d890 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.405+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd00a1b10 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fdc01da90 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 10 client.15441 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564d2ee40aa0 con 0x564d2edbf7d0 2024-04-25T21:09:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd00a1b10 con 0x564d2edbf7d0 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6fdc032c20 con 0x564d2edbf7d0 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 20 client.15441 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2ee45640 0x564d2ee44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 4 client.15441 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 --> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] -- command(tid 0: {"prefix": "status"}) -- 0x564d2ed19570 con 0x564d2ee45640 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.406+0000 7f6fe563e700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2ee45640 0x564d2ee44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6fc97fa700 10 client.15441 ms_handle_connect on v2:172.21.15.98:6838/899178427 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6fe563e700 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2ee45640 0x564d2ee44350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88193c8a41487509 server_cookie=296625d23ce21277 in_seq=0 out_seq=0 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6fc97fa700 1 -- 172.21.15.37:0/1199258294 <== mds.1 v2:172.21.15.98:6838/899178427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564d2ed19570 con 0x564d2ee45640 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6fc97fa700 10 client.15441 handle_command_reply: tid=0 2024-04-25T21:09:06.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 client.15441 shutdown 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 msgr2=0x564d2ede3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x564d2edcc550 tx=0x7f6fdc004750 comp rx=0 tx=0).stop 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 shutdown_connections 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d2edc4580 0x564d2edddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:6838/899178427,v1:172.21.15.98:6839/899178427] conn(0x564d2ee45640 0x564d2ee44350 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564d2edbf7d0 0x564d2ede3700 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 --2- 172.21.15.37:0/1199258294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564d2edc00f0 0x564d2ede3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6fc97fa700 0 client.15441 ms_handle_reset on v2:172.21.15.98:6838/899178427 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 >> 172.21.15.37:0/1199258294 conn(0x564d2ecec640 msgr2=0x564d2edcaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 shutdown_connections 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.407+0000 7f6ff292cb80 1 -- 172.21.15.37:0/1199258294 wait complete. 2024-04-25T21:09:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6ff292cb80 20 client.15441 trim_cache size 0 max 16384 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24458, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 229.667241644, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 260.17898328699999 2024-04-25T21:09:06.412 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 msgr2=0x7f6fcc0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 0x7f6fcc0a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc005fd0 tx=0x7f6fdc005dc0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 msgr2=0x7f6fe019b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6fd00057c0 tx=0x7f6fd00057f0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 shutdown_connections 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6fcc0a3fe0 0x7f6fcc0a64a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fe010f4e0 0x7f6fe019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6fe006afe0 0x7f6fe019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 --2- 172.21.15.37:0/582200442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fe006b900 0x7f6fe019b8e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:06.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 >> 172.21.15.37:0/582200442 conn(0x7f6fe00788b0 msgr2=0x7f6fe0110bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:06.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 shutdown_connections 2024-04-25T21:09:06.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:06.408+0000 7f6fe80a3700 1 -- 172.21.15.37:0/582200442 wait complete. 2024-04-25T21:09:07.204 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T21:09:07.472 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:09:07.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:07 smithi037 ceph-mon[29195]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:09:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:07 smithi177 ceph-mon[33043]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:09:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:07 smithi098 ceph-mon[32569]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:09:09.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- 172.21.15.37:0/689855433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 msgr2=0x7fe180102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 --2- 172.21.15.37:0/689855433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180102f70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe1740099c0 tx=0x7fe174009cd0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- 172.21.15.37:0/689855433 shutdown_connections 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 --2- 172.21.15.37:0/689855433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1801034b0 0x7fe18010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 --2- 172.21.15.37:0/689855433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180107940 0x7fe180107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 --2- 172.21.15.37:0/689855433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180102f70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- 172.21.15.37:0/689855433 >> 172.21.15.37:0/689855433 conn(0x7fe180079ba0 msgr2=0x7fe180079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- 172.21.15.37:0/689855433 shutdown_connections 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- 172.21.15.37:0/689855433 wait complete. 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 Processor -- start 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.282+0000 7fe18660d700 1 -- start start 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801034b0 0x7fe180199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe180107940 0x7fe18019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe180112c30 con 0x7fe180102af0 2024-04-25T21:09:09.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe180112ab0 con 0x7fe180107940 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe18660d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe180112db0 con 0x7fe1801034b0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38530/0 (socket says 172.21.15.37:38530) 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 -- 172.21.15.37:0/2451319623 learned_addr learned my addr 172.21.15.37:0/2451319623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801034b0 0x7fe180199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe184baa700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe180107940 0x7fe18019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 -- 172.21.15.37:0/2451319623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801034b0 msgr2=0x7fe180199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801034b0 0x7fe180199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 -- 172.21.15.37:0/2451319623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe180107940 msgr2=0x7fe18019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe180107940 0x7fe18019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 -- 172.21.15.37:0/2451319623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18019e110 con 0x7fe180102af0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17ffff700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d980 tx=0x7fe17000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.283+0000 7fe17d7fa700 1 -- 172.21.15.37:0/2451319623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170016070 con 0x7fe180102af0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.284+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18019e400 con 0x7fe180102af0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.284+0000 7fe17d7fa700 1 -- 172.21.15.37:0/2451319623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1700044d0 con 0x7fe180102af0 2024-04-25T21:09:09.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.284+0000 7fe17d7fa700 1 -- 172.21.15.37:0/2451319623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170004d00 con 0x7fe180102af0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.284+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1801a5c80 con 0x7fe180102af0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.286+0000 7fe17d7fa700 1 -- 172.21.15.37:0/2451319623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe170004020 con 0x7fe180102af0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.286+0000 7fe17d7fa700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 0x7fe1680a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.286+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 0x7fe1680a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.286+0000 7fe17d7fa700 1 -- 172.21.15.37:0/2451319623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe1700d08f0 con 0x7fe180102af0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 0x7fe1680a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe18007a210 tx=0x7fe17401a040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 Processor -- start 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 -- start start 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5d500 0x556b8cb61450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b8caaace0 con 0x556b8cb61990 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8ca51910 con 0x556b8cb5d500 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe190e96b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8caa6ca0 con 0x556b8cb5cbe0 2024-04-25T21:09:09.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe184baa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.287+0000 7fe184baa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38536/0 (socket says 172.21.15.37:38536) 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 -- 172.21.15.37:0/3110344630 learned_addr learned my addr 172.21.15.37:0/3110344630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5d500 0x556b8cb61450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 -- 172.21.15.37:0/3110344630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 msgr2=0x556b8cb5cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb5cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 -- 172.21.15.37:0/3110344630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5d500 msgr2=0x556b8cb61450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5d500 0x556b8cb61450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 -- 172.21.15.37:0/3110344630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c9f8880 con 0x556b8cb61990 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe184baa700 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe1780085a0 tx=0x7fe1780088b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a0019d091699fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe1667fc700 1 -- 172.21.15.37:0/3110344630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178003c00 con 0x556b8cb61990 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe1667fc700 1 -- 172.21.15.37:0/3110344630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe178003da0 con 0x556b8cb61990 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.288+0000 7fe1667fc700 1 -- 172.21.15.37:0/3110344630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1780127f0 con 0x556b8cb61990 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- 172.21.15.37:0/3110344630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 msgr2=0x556b8cb61df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe1780085a0 tx=0x7fe1780088b0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- 172.21.15.37:0/3110344630 shutdown_connections 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb61990 0x556b8cb61df0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb5cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3110344630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5d500 0x556b8cb61450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- 172.21.15.37:0/3110344630 >> 172.21.15.37:0/3110344630 conn(0x556b8ca85640 msgr2=0x556b8cb53820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- 172.21.15.37:0/3110344630 shutdown_connections 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- 172.21.15.37:0/3110344630 wait complete. 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.289+0000 7fe190e96b80 1 -- start start 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 Processor -- start 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5cbe0 0x556b8cb68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 0x556b8cb66690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb69a20 con 0x556b8cb5d500 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8ca49b80 con 0x556b8cb5cbe0 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb69ba0 con 0x556b8cb61990 2024-04-25T21:09:09.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe190e96b80 10 client.0 _ll_register_callbacks cb 0x556b8ca43900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38546/0 (socket says 172.21.15.37:38546) 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 -- 172.21.15.37:0/3817218264 learned_addr learned my addr 172.21.15.37:0/3817218264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5cbe0 0x556b8cb68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe184baa700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 0x556b8cb66690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 -- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 msgr2=0x556b8cb66690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 0x556b8cb66690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 -- 172.21.15.37:0/3817218264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5cbe0 msgr2=0x556b8cb68e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5cbe0 0x556b8cb68e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 -- 172.21.15.37:0/3817218264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8ca62220 con 0x556b8cb5d500 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe184baa700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 0x556b8cb66690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe174015000 tx=0x7fe174009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd25a12f8e9357d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.290+0000 7fe15f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174034060 con 0x556b8cb5d500 2024-04-25T21:09:09.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe174023820 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe178003da0 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe17400be70 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 10 client.15453 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b8cb74250 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe178003da0 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe17402fb00 con 0x556b8cb5d500 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 10 client.15453 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 20 client.15453 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cb77d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 4 client.15453 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.291+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8cab84d0 con 0x556b8cb768e0 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.292+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cb77d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.292+0000 7fe15f7fe700 10 client.15453 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.292+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cb77d50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52c5d161277f163 server_cookie=79a1705f452d1f1d in_seq=0 out_seq=0 2024-04-25T21:09:09.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3817218264 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b8cab84d0 con 0x556b8cb768e0 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe15f7fe700 10 client.15453 handle_command_reply: tid=0 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 client.15453 shutdown 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 msgr2=0x556b8cb66150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe174015000 tx=0x7fe174009cd0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 shutdown_connections 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cb77d50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe15f7fe700 0 client.15453 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb61990 0x556b8cb66690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb5cbe0 0x556b8cb68e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3817218264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb66150 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 >> 172.21.15.37:0/3817218264 conn(0x556b8ca85640 msgr2=0x556b8cb52a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 shutdown_connections 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 1 -- 172.21.15.37:0/3817218264 wait complete. 2024-04-25T21:09:09.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.293+0000 7fe190e96b80 20 client.15453 trim_cache size 0 max 16384 2024-04-25T21:09:09.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.314+0000 7fe190e96b80 1 Processor -- start 2024-04-25T21:09:09.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.314+0000 7fe190e96b80 1 -- start start 2024-04-25T21:09:09.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb57c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb68940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb76090 con 0x556b8cb5d500 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb76760 con 0x556b8cb61990 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe190e96b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb76e30 con 0x556b8cb5cbe0 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38550/0 (socket says 172.21.15.37:38550) 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 -- 172.21.15.37:0/2033295214 learned_addr learned my addr 172.21.15.37:0/2033295214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17ffff700 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb57c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe184baa700 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb68940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 -- 172.21.15.37:0/2033295214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 msgr2=0x556b8cb57c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb57c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 -- 172.21.15.37:0/2033295214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 msgr2=0x556b8cb68940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb68940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 -- 172.21.15.37:0/2033295214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8cb617c0 con 0x556b8cb5d500 2024-04-25T21:09:09.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.315+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe1740233e0 tx=0x7fe174005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb40b1e5bbbeebc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe165ffb700 1 -- 172.21.15.37:0/2033295214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174034040 con 0x556b8cb5d500 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe165ffb700 1 -- 172.21.15.37:0/2033295214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1740176c0 con 0x556b8cb5d500 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe165ffb700 1 -- 172.21.15.37:0/2033295214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17402fb20 con 0x556b8cb5d500 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 -- 172.21.15.37:0/2033295214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 msgr2=0x556b8cb64db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe1740233e0 tx=0x7fe174005fd0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 -- 172.21.15.37:0/2033295214 shutdown_connections 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb68940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5cbe0 0x556b8cb57c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 --2- 172.21.15.37:0/2033295214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5d500 0x556b8cb64db0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.316+0000 7fe190e96b80 1 -- 172.21.15.37:0/2033295214 >> 172.21.15.37:0/2033295214 conn(0x556b8ca85640 msgr2=0x556b8cb726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- 172.21.15.37:0/2033295214 shutdown_connections 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- 172.21.15.37:0/2033295214 wait complete. 2024-04-25T21:09:09.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- start start 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 Processor -- start 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5d500 0x556b8cb81710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb77bd0 con 0x556b8cb5cbe0 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb77500 con 0x556b8cb61990 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.317+0000 7fe190e96b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556b8cb782a0 con 0x556b8cb5d500 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe190e96b80 10 client.0 _ll_register_callbacks cb 0x556b8ca43900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38556/0 (socket says 172.21.15.37:38556) 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 -- 172.21.15.37:0/3323067957 learned_addr learned my addr 172.21.15.37:0/3323067957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe184baa700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5d500 0x556b8cb81710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 -- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5d500 msgr2=0x556b8cb81710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5d500 0x556b8cb81710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 -- 172.21.15.37:0/3323067957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 msgr2=0x556b8cb7a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb7a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 -- 172.21.15.37:0/3323067957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8cb7ad90 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe184baa700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb7a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe17ffff700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe17009c380 tx=0x7fe17009c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667427770dc6febc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe15f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17009c680 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe1700b5070 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1740176c0 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.318+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1700a0a30 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 10 client.15459 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556b8cbde890 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1740176c0 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe1700a4e60 con 0x556b8cb5cbe0 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:09.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 10 client.15459 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 20 client.15459 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cbe2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 4 client.15459 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "status"}) -- 0x556b8cab2570 con 0x556b8cb768e0 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.319+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cbe2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe15f7fe700 10 client.15459 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:09.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe17f7fe700 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cbe2130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2340201603a809 server_cookie=f6e0219e8e01a2af in_seq=0 out_seq=0 2024-04-25T21:09:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe15f7fe700 1 -- 172.21.15.37:0/3323067957 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556b8cab2570 con 0x556b8cb768e0 2024-04-25T21:09:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe15f7fe700 10 client.15459 handle_command_reply: tid=0 2024-04-25T21:09:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 client.15459 shutdown 2024-04-25T21:09:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 msgr2=0x556b8cb811d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe17009c380 tx=0x7fe17009c580 comp rx=0 tx=0).stop 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 shutdown_connections 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556b8cb768e0 0x556b8cbe2130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556b8cb61990 0x556b8cb7a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556b8cb5cbe0 0x556b8cb811d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 --2- 172.21.15.37:0/3323067957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556b8cb5d500 0x556b8cb81710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 >> 172.21.15.37:0/3323067957 conn(0x556b8ca85640 msgr2=0x556b8cb60280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe15f7fe700 0 client.15459 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 shutdown_connections 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.320+0000 7fe190e96b80 1 -- 172.21.15.37:0/3323067957 wait complete. 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe190e96b80 20 client.15459 trim_cache size 0 max 16384 2024-04-25T21:09:09.324 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "1788dafa-0346-11ef-bc93-c7b262605968", 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24469, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 244.657440466, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 27, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 273.270983785 2024-04-25T21:09:09.325 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 msgr2=0x7fe1680a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 0x7fe1680a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe18007a210 tx=0x7fe17401a040 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 msgr2=0x7fe180199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe17000d980 tx=0x7fe17000dd40 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 shutdown_connections 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1680a3f60 0x7fe1680a6420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe180107940 0x7fe18019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe180102af0 0x7fe180199100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 --2- 172.21.15.37:0/2451319623 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801034b0 0x7fe180199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 >> 172.21.15.37:0/2451319623 conn(0x7fe180079ba0 msgr2=0x7fe180100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:09.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 shutdown_connections 2024-04-25T21:09:09.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:09.321+0000 7fe18660d700 1 -- 172.21.15.37:0/2451319623 wait complete. 2024-04-25T21:09:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:09 smithi037 ceph-mon[29195]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:09 smithi177 ceph-mon[33043]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:10.143 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T21:09:10.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:09 smithi098 ceph-mon[32569]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- 172.21.15.37:0/2015202323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 msgr2=0x7fa7201025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 --2- 172.21.15.37:0/2015202323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201025b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa710009900 tx=0x7fa71002a410 comp rx=0 tx=0).stop 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- 172.21.15.37:0/2015202323 shutdown_connections 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 --2- 172.21.15.37:0/2015202323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7201072a0 0x7fa72010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 --2- 172.21.15.37:0/2015202323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa720102b80 0x7fa720106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 --2- 172.21.15.37:0/2015202323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201025b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- 172.21.15.37:0/2015202323 >> 172.21.15.37:0/2015202323 conn(0x7fa7200793e0 msgr2=0x7fa7200797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- 172.21.15.37:0/2015202323 shutdown_connections 2024-04-25T21:09:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- 172.21.15.37:0/2015202323 wait complete. 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 Processor -- start 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.235+0000 7fa725b4f640 1 -- start start 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7201072a0 0x7fa720198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72019cf80 0x7fa72019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa720112200 con 0x7fa7201021d0 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa720112080 con 0x7fa72019cf80 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa725b4f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa720112380 con 0x7fa7201072a0 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38574/0 (socket says 172.21.15.37:38574) 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 -- 172.21.15.37:0/53606535 learned_addr learned my addr 172.21.15.37:0/53606535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71effd640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7201072a0 0x7fa720198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71ffff640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72019cf80 0x7fa72019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 -- 172.21.15.37:0/53606535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7201072a0 msgr2=0x7fa720198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7201072a0 0x7fa720198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 -- 172.21.15.37:0/53606535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72019cf80 msgr2=0x7fa72019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72019cf80 0x7fa72019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 -- 172.21.15.37:0/53606535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72019da70 con 0x7fa7201021d0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa710005940 tx=0x7fa7100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.236+0000 7fa71cff9640 1 -- 172.21.15.37:0/53606535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71003f030 con 0x7fa7201021d0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.237+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72019dd00 con 0x7fa7201021d0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.237+0000 7fa71cff9640 1 -- 172.21.15.37:0/53606535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa710003cb0 con 0x7fa7201021d0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.237+0000 7fa71cff9640 1 -- 172.21.15.37:0/53606535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710035070 con 0x7fa7201021d0 2024-04-25T21:09:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.237+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7201a5520 con 0x7fa7201021d0 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa71cff9640 1 -- 172.21.15.37:0/53606535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa710042030 con 0x7fa7201021d0 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 Processor -- start 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 -- start start 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 0x563d3334ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d333572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d332b1970 con 0x563d33357800 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d3324e4a0 con 0x563d3334c620 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa72886e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563d332ad930 con 0x563d3334cf40 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.239+0000 7fa71cff9640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 0x7fa7080a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38580/0 (socket says 172.21.15.37:38580) 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 -- 172.21.15.37:0/3426040258 learned_addr learned my addr 172.21.15.37:0/3426040258 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d333572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71ffff640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 0x7fa7080a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71cff9640 1 -- 172.21.15.37:0/53606535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa7100f12a0 con 0x7fa7201021d0 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71ffff640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 0x563d3334ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 -- 172.21.15.37:0/3426040258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 msgr2=0x563d333572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d333572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 -- 172.21.15.37:0/3426040258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 msgr2=0x563d3334ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 0x563d3334ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 -- 172.21.15.37:0/3426040258 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d32fc4210 con 0x563d33357800 2024-04-25T21:09:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d333572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71ffff640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 0x563d3334ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71effd640 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b6c0 tx=0x7fa71400bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2860048ad06d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa71ffff640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 0x7fa7080a6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa718009800 tx=0x7fa718009790 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa705ffb640 1 -- 172.21.15.37:0/3426040258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714003bd0 con 0x563d33357800 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa705ffb640 1 -- 172.21.15.37:0/3426040258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa714015070 con 0x563d33357800 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.240+0000 7fa705ffb640 1 -- 172.21.15.37:0/3426040258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714004cd0 con 0x563d33357800 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- 172.21.15.37:0/3426040258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 msgr2=0x563d33359bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa71400b6c0 tx=0x7fa71400bb80 comp rx=0 tx=0).stop 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- 172.21.15.37:0/3426040258 shutdown_connections 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d33357800 0x563d33359bf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d333572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 --2- 172.21.15.37:0/3426040258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d3334c620 0x563d3334ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- 172.21.15.37:0/3426040258 >> 172.21.15.37:0/3426040258 conn(0x563d33348b30 msgr2=0x563d33348f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- 172.21.15.37:0/3426040258 shutdown_connections 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- 172.21.15.37:0/3426040258 wait complete. 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.241+0000 7fa72886e740 1 -- start start 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 Processor -- start 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 0x563d33357060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33357800 0x563d33354890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d3335a430 con 0x563d3334cf40 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563d3324c0b0 con 0x563d33357800 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563d3335a5b0 con 0x563d3334c620 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa72886e740 10 client.0 _ll_register_callbacks cb 0x563d33239d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38584/0 (socket says 172.21.15.37:38584) 2024-04-25T21:09:10.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 -- 172.21.15.37:0/3672213176 learned_addr learned my addr 172.21.15.37:0/3672213176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71ffff640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33357800 0x563d33354890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 0x563d33357060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 -- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 msgr2=0x563d33357060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 0x563d33357060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 -- 172.21.15.37:0/3672213176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33357800 msgr2=0x563d33354890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33357800 0x563d33354890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 -- 172.21.15.37:0/3672213176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d33090620 con 0x563d3334cf40 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 0x563d33357060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa71effd640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa714005cd0 tx=0x7fa714003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8da55d12909cddd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.242+0000 7fa6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714028060 con 0x563d3334cf40 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa714011850 con 0x563d3334cf40 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa714015070 con 0x563d3334cf40 2024-04-25T21:09:10.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa714011a50 con 0x563d3334cf40 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 10 client.15471 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d333661c0 con 0x563d3334cf40 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa714015070 con 0x563d3334cf40 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa714022070 con 0x563d3334cf40 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 10 client.15471 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 20 client.15471 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33367570 0x563d33369960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 4 client.15471 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d332bf390 con 0x563d33367570 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.243+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33367570 0x563d33369960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.244+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33367570 0x563d33369960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77682dad537ebf61 server_cookie=e15d0f09439bace7 in_seq=0 out_seq=0 2024-04-25T21:09:10.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.244+0000 7fa6feffd640 10 client.15471 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa6feffd640 1 -- 172.21.15.37:0/3672213176 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d332bf390 con 0x563d33367570 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa6feffd640 10 client.15471 handle_command_reply: tid=0 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 client.15471 shutdown 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 msgr2=0x563d33354350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa714005cd0 tx=0x7fa714003e00 comp rx=0 tx=0).stop 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 shutdown_connections 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33367570 0x563d33369960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33357800 0x563d33354890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334cf40 0x563d33354350 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa6feffd640 0 client.15471 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 --2- 172.21.15.37:0/3672213176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334c620 0x563d33357060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 >> 172.21.15.37:0/3672213176 conn(0x563d33348b30 msgr2=0x563d3334e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 shutdown_connections 2024-04-25T21:09:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 1 -- 172.21.15.37:0/3672213176 wait complete. 2024-04-25T21:09:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.245+0000 7fa72886e740 20 client.15471 trim_cache size 0 max 16384 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 Processor -- start 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 -- start start 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d333632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d33356b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201ada60 con 0x563d3334c620 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201af5a0 con 0x563d33348b30 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa72886e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201ae130 con 0x563d3334cf40 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38594/0 (socket says 172.21.15.37:38594) 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.260+0000 7fa71effd640 1 -- 172.21.15.37:0/1473702990 learned_addr learned my addr 172.21.15.37:0/1473702990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:10.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71ffff640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d33356b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d333632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 -- 172.21.15.37:0/1473702990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 msgr2=0x563d33356b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d33356b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 -- 172.21.15.37:0/1473702990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 msgr2=0x563d333632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d333632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 -- 172.21.15.37:0/1473702990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d33348960 con 0x563d3334c620 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d333632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa71effd640 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa7140045c0 tx=0x7fa71400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141ea20c7dc79367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa7057fa640 1 -- 172.21.15.37:0/1473702990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714028070 con 0x563d3334c620 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa7057fa640 1 -- 172.21.15.37:0/1473702990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa714015050 con 0x563d3334c620 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.261+0000 7fa7057fa640 1 -- 172.21.15.37:0/1473702990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71401ae60 con 0x563d3334c620 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- 172.21.15.37:0/1473702990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 msgr2=0x563d3334f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa7140045c0 tx=0x7fa71400bfd0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- 172.21.15.37:0/1473702990 shutdown_connections 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d33356b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f970 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 --2- 172.21.15.37:0/1473702990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d333632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- 172.21.15.37:0/1473702990 >> 172.21.15.37:0/1473702990 conn(0x563d333549f0 msgr2=0x563d3334f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- 172.21.15.37:0/1473702990 shutdown_connections 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- 172.21.15.37:0/1473702990 wait complete. 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 -- start start 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.262+0000 7fa72886e740 1 Processor -- start 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d3334eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d3334f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201aeed0 con 0x563d3334c620 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201ae800 con 0x563d33348b30 2024-04-25T21:09:10.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201aba60 con 0x563d3334cf40 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 10 client.0 _ll_register_callbacks cb 0x7fa714004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38608/0 (socket says 172.21.15.37:38608) 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 -- 172.21.15.37:0/3770690611 learned_addr learned my addr 172.21.15.37:0/3770690611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d3334eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71ffff640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d3334f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 -- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 msgr2=0x563d3334f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d3334f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 -- 172.21.15.37:0/3770690611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 msgr2=0x563d3334eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d3334eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 -- 172.21.15.37:0/3770690611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3334d850 con 0x563d3334c620 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa71effd640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa714004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa37369b750b2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714028040 con 0x563d3334c620 2024-04-25T21:09:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa714015050 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.263+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa714019620 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa714047050 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 10 client.15477 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d333cd8e0 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa714015050 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa71403e070 con 0x563d3334c620 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 20 client.15477 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33357800 0x563d333cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 4 client.15477 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d332b9200 con 0x563d33357800 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.264+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33357800 0x563d333cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:10.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.265+0000 7fa6feffd640 10 client.15477 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.265+0000 7fa71f7fe640 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33357800 0x563d333cff50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac90c07374557a16 server_cookie=aba62ff0289468a2 in_seq=0 out_seq=0 2024-04-25T21:09:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.272+0000 7fa6feffd640 1 -- 172.21.15.37:0/3770690611 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d332b9200 con 0x563d33357800 2024-04-25T21:09:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.272+0000 7fa6feffd640 10 client.15477 handle_command_reply: tid=0 2024-04-25T21:09:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.272+0000 7fa72886e740 1 client.15477 shutdown 2024-04-25T21:09:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 msgr2=0x563d3334f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa714000c00 tx=0x7fa714004710 comp rx=0 tx=0).stop 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 shutdown_connections 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563d33357800 0x563d333cff50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563d3334cf40 0x563d3334f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d3334c620 0x563d3334f3e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa6feffd640 0 client.15477 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 --2- 172.21.15.37:0/3770690611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563d33348b30 0x563d3334eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 >> 172.21.15.37:0/3770690611 conn(0x563d333549f0 msgr2=0x563d33347cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 shutdown_connections 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 1 -- 172.21.15.37:0/3770690611 wait complete. 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa72886e740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T21:09:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:09:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:09:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:09:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:09:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 msgr2=0x7fa7080a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 0x7fa7080a6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa718009800 tx=0x7fa718009790 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 msgr2=0x7fa7201986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa710005940 tx=0x7fa7100042f0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 shutdown_connections 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72019cf80 0x7fa72019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa7080a3f70 0x7fa7080a6430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7201072a0 0x7fa720198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 --2- 172.21.15.37:0/53606535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7201021d0 0x7fa7201986c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.273+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 >> 172.21.15.37:0/53606535 conn(0x7fa7200793e0 msgr2=0x7fa7200ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.274+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 shutdown_connections 2024-04-25T21:09:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:10.274+0000 7fa725b4f640 1 -- 172.21.15.37:0/53606535 wait complete. 2024-04-25T21:09:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:11 smithi037 ceph-mon[29195]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-04-25T21:09:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:11 smithi177 ceph-mon[33043]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-04-25T21:09:12.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:11 smithi098 ceph-mon[32569]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-04-25T21:09:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:13 smithi037 ceph-mon[29195]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:09:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:13 smithi177 ceph-mon[33043]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:09:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:13 smithi098 ceph-mon[32569]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-25T21:09:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:15 smithi037 ceph-mon[29195]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:15 smithi177 ceph-mon[33043]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:16.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:15 smithi098 ceph-mon[32569]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-04-25T21:09:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:17 smithi037 ceph-mon[29195]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T21:09:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:17 smithi177 ceph-mon[33043]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T21:09:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:17 smithi098 ceph-mon[32569]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-25T21:09:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:19 smithi037 ceph-mon[29195]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:19 smithi177 ceph-mon[33043]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:19 smithi098 ceph-mon[32569]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:21 smithi037 ceph-mon[29195]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:21 smithi177 ceph-mon[33043]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:22.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:21 smithi098 ceph-mon[32569]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- 172.21.15.37:0/4152151607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 msgr2=0x7f34a8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 --2- 172.21.15.37:0/4152151607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8103440 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009900 tx=0x7f34a402a480 comp rx=0 tx=0).stop 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- 172.21.15.37:0/4152151607 shutdown_connections 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 --2- 172.21.15.37:0/4152151607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a810c580 0x7f34a810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 --2- 172.21.15.37:0/4152151607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a8103980 0x7f34a810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 --2- 172.21.15.37:0/4152151607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8103440 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- 172.21.15.37:0/4152151607 >> 172.21.15.37:0/4152151607 conn(0x7f34a8077fd0 msgr2=0x7f34a80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- 172.21.15.37:0/4152151607 shutdown_connections 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- 172.21.15.37:0/4152151607 wait complete. 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 Processor -- start 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.363+0000 7f34af344640 1 -- start start 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 0x7f34a8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a810c580 0x7f34a819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81121b0 con 0x7f34a8103060 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8112030 con 0x7f34a8103980 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34af344640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8112330 con 0x7f34a810c580 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51588/0 (socket says 172.21.15.37:51588) 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 -- 172.21.15.37:0/1642862380 learned_addr learned my addr 172.21.15.37:0/1642862380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ac8df640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 0x7f34a8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 -- 172.21.15.37:0/1642862380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a810c580 msgr2=0x7f34a819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a810c580 0x7f34a819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a810c580 0x7f34a819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 -- 172.21.15.37:0/1642862380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 msgr2=0x7f34a8198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 0x7f34a8198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 -- 172.21.15.37:0/1642862380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a819d720 con 0x7f34a8103060 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ac8df640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 0x7f34a8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005940 tx=0x7f34a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.364+0000 7f349e7fc640 1 -- 172.21.15.37:0/1642862380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a403f030 con 0x7f34a8103060 2024-04-25T21:09:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.365+0000 7f349e7fc640 1 -- 172.21.15.37:0/1642862380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a4003aa0 con 0x7f34a8103060 2024-04-25T21:09:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.365+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a819d9b0 con 0x7f34a8103060 2024-04-25T21:09:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.365+0000 7f349e7fc640 1 -- 172.21.15.37:0/1642862380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4035070 con 0x7f34a8103060 2024-04-25T21:09:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.365+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81a52a0 con 0x7f34a8103060 2024-04-25T21:09:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.367+0000 7f34b205e740 1 Processor -- start 2024-04-25T21:09:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.367+0000 7f34b205e740 1 -- start start 2024-04-25T21:09:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.367+0000 7f349e7fc640 1 -- 172.21.15.37:0/1642862380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f34a4042030 con 0x7f34a8103060 2024-04-25T21:09:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 0x555ec80cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555ec8023970 con 0x555ec80be620 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555ec7fc04a0 con 0x555ec80c9800 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34b205e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ec801f930 con 0x555ec80bef40 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51592/0 (socket says 172.21.15.37:51592) 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 -- 172.21.15.37:0/645926453 learned_addr learned my addr 172.21.15.37:0/645926453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ac8df640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 0x555ec80cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 -- 172.21.15.37:0/645926453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 msgr2=0x555ec80c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 -- 172.21.15.37:0/645926453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 msgr2=0x555ec80cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 0x555ec80cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 -- 172.21.15.37:0/645926453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec7d36210 con 0x555ec80be620 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ac8df640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 0x555ec80cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f349e7fc640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 0x7f34940a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 0x7f34940a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.368+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009df0 tx=0x7f34a000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60572e69f5f5120f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f349e7fc640 1 -- 172.21.15.37:0/1642862380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f34a40f1220 con 0x7f34a8103060 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34937fe640 1 -- 172.21.15.37:0/645926453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016070 con 0x555ec80be620 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34937fe640 1 -- 172.21.15.37:0/645926453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0005020 con 0x555ec80be620 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34937fe640 1 -- 172.21.15.37:0/645926453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00116d0 con 0x555ec80be620 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34b205e740 1 -- 172.21.15.37:0/645926453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 msgr2=0x555ec80bea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34ad0e0640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 0x7f34940a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34a40ee5f0 tx=0x7f34a40f6040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.369+0000 7f34b205e740 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009df0 tx=0x7f34a000e6c0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 -- 172.21.15.37:0/645926453 shutdown_connections 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80c9800 0x555ec80cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- 172.21.15.37:0/645926453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80be620 0x555ec80bea00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 -- 172.21.15.37:0/645926453 >> 172.21.15.37:0/645926453 conn(0x555ec80bab30 msgr2=0x555ec80baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 -- 172.21.15.37:0/645926453 shutdown_connections 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 -- 172.21.15.37:0/645926453 wait complete. 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 -- start start 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 Processor -- start 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80be620 0x555ec80c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80bef40 0x555ec80c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.370+0000 7f34b205e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34b205e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555ec80cc430 con 0x555ec80c9800 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34b205e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555ec7fbe0b0 con 0x555ec80bef40 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34b205e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555ec80cc5b0 con 0x555ec80be620 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34b205e740 10 client.0 _ll_register_callbacks cb 0x555ec7fabd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51598/0 (socket says 172.21.15.37:51598) 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 -- 172.21.15.37:0/1836814452 learned_addr learned my addr 172.21.15.37:0/1836814452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80be620 0x555ec80c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ac8df640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80bef40 0x555ec80c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 -- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80be620 msgr2=0x555ec80c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80be620 0x555ec80c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 -- 172.21.15.37:0/1836814452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80bef40 msgr2=0x555ec80c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80bef40 0x555ec80c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 -- 172.21.15.37:0/1836814452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec7e02620 con 0x555ec80c9800 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005b40 tx=0x7f34a0005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858e11b74e24186f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f3490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016070 con 0x555ec80c9800 2024-04-25T21:09:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a0005020 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f34a0005620 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.371+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34a0011470 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 10 client.15489 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ec80d8260 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0005020 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f34a000f040 con 0x555ec80c9800 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 20 client.15489 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80d96d0 0x555ec80dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 4 client.15489 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ec8031390 con 0x555ec80d96d0 2024-04-25T21:09:23.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.372+0000 7f34ac8df640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80d96d0 0x555ec80dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f34ac8df640 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80d96d0 0x555ec80dbac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a128f7600c705440 server_cookie=f13589279eaca579 in_seq=0 out_seq=0 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f3490ff9640 10 client.15489 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f3490ff9640 1 -- 172.21.15.37:0/1836814452 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ec8031390 con 0x555ec80d96d0 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f3490ff9640 10 client.15489 handle_command_reply: tid=0 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f34b205e740 1 client.15489 shutdown 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 msgr2=0x555ec80c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.373+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f34a0005b40 tx=0x7f34a0005910 comp rx=0 tx=0).stop 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 shutdown_connections 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80d96d0 0x555ec80dbac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80c9800 0x555ec80c6900 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80bef40 0x555ec80c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f3490ff9640 0 client.15489 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 --2- 172.21.15.37:0/1836814452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80be620 0x555ec80c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 >> 172.21.15.37:0/1836814452 conn(0x555ec80bab30 msgr2=0x555ec80c0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 shutdown_connections 2024-04-25T21:09:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 1 -- 172.21.15.37:0/1836814452 wait complete. 2024-04-25T21:09:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.374+0000 7f34b205e740 20 client.15489 trim_cache size 0 max 16384 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 Processor -- start 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 -- start start 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81adbc0 con 0x555ec80bab30 2024-04-25T21:09:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81af700 con 0x555ec80be620 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34b205e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81ae290 con 0x555ec80bef40 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51606/0 (socket says 172.21.15.37:51606) 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 -- 172.21.15.37:0/513847632 learned_addr learned my addr 172.21.15.37:0/513847632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ac8df640 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 -- 172.21.15.37:0/513847632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 msgr2=0x555ec80c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 -- 172.21.15.37:0/513847632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 msgr2=0x555ec80c1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.389+0000 7f34ad8e1640 1 -- 172.21.15.37:0/513847632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec80bfa10 con 0x555ec80bab30 2024-04-25T21:09:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f34a0002230 tx=0x7f34a0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add17612edfa99fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f3492ffd640 1 -- 172.21.15.37:0/513847632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016070 con 0x555ec80bab30 2024-04-25T21:09:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f3492ffd640 1 -- 172.21.15.37:0/513847632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0037040 con 0x555ec80bab30 2024-04-25T21:09:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f3492ffd640 1 -- 172.21.15.37:0/513847632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0036070 con 0x555ec80bab30 2024-04-25T21:09:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f34b205e740 1 -- 172.21.15.37:0/513847632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 msgr2=0x555ec80d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.390+0000 7f34b205e740 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f34a0002230 tx=0x7f34a0009580 comp rx=0 tx=0).stop 2024-04-25T21:09:23.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- 172.21.15.37:0/513847632 shutdown_connections 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bef40 0x555ec80c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- 172.21.15.37:0/513847632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bab30 0x555ec80d52f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- 172.21.15.37:0/513847632 >> 172.21.15.37:0/513847632 conn(0x555ec80c6a60 msgr2=0x555ec80c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- 172.21.15.37:0/513847632 shutdown_connections 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- 172.21.15.37:0/513847632 wait complete. 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- start start 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 Processor -- start 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bab30 0x555ec80d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81af030 con 0x555ec80bef40 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81ae960 con 0x555ec80be620 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81abba0 con 0x555ec80bab30 2024-04-25T21:09:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.391+0000 7f34b205e740 10 client.0 _ll_register_callbacks cb 0x7f34a0005260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51614/0 (socket says 172.21.15.37:51614) 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 -- 172.21.15.37:0/3680026080 learned_addr learned my addr 172.21.15.37:0/3680026080 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ac8df640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bab30 0x555ec80d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 -- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bab30 msgr2=0x555ec80d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bab30 0x555ec80d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 -- 172.21.15.37:0/3680026080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 msgr2=0x555ec80c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 -- 172.21.15.37:0/3680026080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec80c3c70 con 0x555ec80bef40 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ac8df640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34ad8e1640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ed40 tx=0x7f34a0004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba68182ba6784d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f3490ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0016070 con 0x555ec80bef40 2024-04-25T21:09:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a0037040 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f34a0029a00 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f34a0036040 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34b205e740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.392+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ec813fa70 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0037040 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f34a003a020 con 0x555ec80bef40 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 20 client.15495 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80c9800 0x555ec8142080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 4 client.15495 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ec802b200 con 0x555ec80c9800 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34ac8df640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80c9800 0x555ec8142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.394+0000 7f3490ff9640 10 client.15495 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.393+0000 7f34ac8df640 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80c9800 0x555ec8142080 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc89d0a4a59ddf58 server_cookie=53f8ca48956eef30 in_seq=0 out_seq=0 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.397+0000 7f3490ff9640 1 -- 172.21.15.37:0/3680026080 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ec802b200 con 0x555ec80c9800 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.397+0000 7f3490ff9640 10 client.15495 handle_command_reply: tid=0 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 client.15495 shutdown 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 msgr2=0x555ec80c33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f34a000ed40 tx=0x7f34a0004ef0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 shutdown_connections 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555ec80c9800 0x555ec8142080 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555ec80bef40 0x555ec80c33f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555ec80be620 0x555ec80c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 --2- 172.21.15.37:0/3680026080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555ec80bab30 0x555ec80d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f3490ff9640 0 client.15495 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 >> 172.21.15.37:0/3680026080 conn(0x555ec80c6a60 msgr2=0x555ec80c48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 shutdown_connections 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 1 -- 172.21.15.37:0/3680026080 wait complete. 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34b205e740 20 client.15495 trim_cache size 0 max 16384 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:09:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:09:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 msgr2=0x7f34940a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 0x7f34940a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34a40ee5f0 tx=0x7f34a40f6040 comp rx=0 tx=0).stop 2024-04-25T21:09:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 msgr2=0x7f34a8198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005940 tx=0x7f34a4004270 comp rx=0 tx=0).stop 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 shutdown_connections 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f34940a3f70 0x7f34940a6430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34a810c580 0x7f34a819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34a8103980 0x7f34a8198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 --2- 172.21.15.37:0/1642862380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34a8103060 0x7f34a8198730 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.398+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 >> 172.21.15.37:0/1642862380 conn(0x7f34a8077fd0 msgr2=0x7f34a80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.399+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 shutdown_connections 2024-04-25T21:09:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:23.399+0000 7f34af344640 1 -- 172.21.15.37:0/1642862380 wait complete. 2024-04-25T21:09:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:23 smithi177 ceph-mon[33043]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:23 smithi098 ceph-mon[32569]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:24.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:23 smithi037 ceph-mon[29195]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:25.144 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:09:25.144 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T21:09:25.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:09:25.428 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:09:25.810 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:25 smithi037 ceph-mon[29195]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T21:09:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:25 smithi177 ceph-mon[33043]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T21:09:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:25 smithi098 ceph-mon[32569]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-25T21:09:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:27 smithi177 ceph-mon[33043]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T21:09:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:27 smithi098 ceph-mon[32569]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T21:09:28.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:27 smithi037 ceph-mon[29195]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-25T21:09:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:29 smithi177 ceph-mon[33043]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:09:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:29 smithi098 ceph-mon[32569]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:09:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:29 smithi037 ceph-mon[29195]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 -- 172.21.15.37:0/2634350580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 msgr2=0x7efeec06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 --2- 172.21.15.37:0/2634350580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec06b440 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efed8009990 tx=0x7efed8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 -- 172.21.15.37:0/2634350580 shutdown_connections 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 --2- 172.21.15.37:0/2634350580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec06b980 0x7efeec10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 --2- 172.21.15.37:0/2634350580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec108a20 0x7efeec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 --2- 172.21.15.37:0/2634350580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec06b440 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 -- 172.21.15.37:0/2634350580 >> 172.21.15.37:0/2634350580 conn(0x7efeec0788b0 msgr2=0x7efeec078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:31.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 -- 172.21.15.37:0/2634350580 shutdown_connections 2024-04-25T21:09:31.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.923+0000 7efef2c76700 1 -- 172.21.15.37:0/2634350580 wait complete. 2024-04-25T21:09:31.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 Processor -- start 2024-04-25T21:09:31.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 -- start start 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec108a20 0x7efeec19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112bc0 con 0x7efeec06afe0 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112a40 con 0x7efeec108a20 2024-04-25T21:09:31.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef2c76700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efeec112d40 con 0x7efeec06b980 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efef0a12700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.924+0000 7efeebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42736/0 (socket says 172.21.15.37:42736) 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 -- 172.21.15.37:0/3618815452 learned_addr learned my addr 172.21.15.37:0/3618815452 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efef1213700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec108a20 0x7efeec19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 -- 172.21.15.37:0/3618815452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec108a20 msgr2=0x7efeec19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec108a20 0x7efeec19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 -- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 msgr2=0x7efeec1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 -- 172.21.15.37:0/3618815452 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec19e090 con 0x7efeec06b980 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efef0a12700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efeebfff700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efed8000c00 tx=0x7efed80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed801f6d0 con 0x7efeec06b980 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec19e320 con 0x7efeec06b980 2024-04-25T21:09:31.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.925+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed801fd20 con 0x7efeec06b980 2024-04-25T21:09:31.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.926+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed800f460 con 0x7efeec06b980 2024-04-25T21:09:31.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.926+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec101930 con 0x7efeec06b980 2024-04-25T21:09:31.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.927+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeec069730 con 0x7efeec06b980 2024-04-25T21:09:31.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.928+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7efed8004260 con 0x7efeec06b980 2024-04-25T21:09:31.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.929+0000 7efee9ffb700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 0x7efedc0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:31.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.929+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7efed80d5c10 con 0x7efeec06b980 2024-04-25T21:09:31.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.929+0000 7efef0a12700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 0x7efedc0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:31.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.929+0000 7efef0a12700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 0x7efedc0a6420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efee000afa0 tx=0x7efee000a380 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:31.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:31.934+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efed8013070 con 0x7efeec06b980 2024-04-25T21:09:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:31 smithi177 ceph-mon[33043]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-25T21:09:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:31 smithi098 ceph-mon[32569]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-25T21:09:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:31 smithi037 ceph-mon[29195]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-25T21:09:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.238+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeec19a3e0 con 0x7efeec06b980 2024-04-25T21:09:32.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.239+0000 7efee9ffb700 1 -- 172.21.15.37:0/3618815452 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7efed80a2240 con 0x7efeec06b980 2024-04-25T21:09:32.239 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:09:32.239 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.241+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 msgr2=0x7efedc0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.241+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 0x7efedc0a6420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efee000afa0 tx=0x7efee000a380 comp rx=0 tx=0).stop 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.241+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 msgr2=0x7efeec199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.241+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efed8000c00 tx=0x7efed80047b0 comp rx=0 tx=0).stop 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 shutdown_connections 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efedc0a3f60 0x7efedc0a6420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efeec108a20 0x7efeec19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeec06afe0 0x7efeec1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 --2- 172.21.15.37:0/3618815452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efeec06b980 0x7efeec199610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 >> 172.21.15.37:0/3618815452 conn(0x7efeec0788b0 msgr2=0x7efeec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 shutdown_connections 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:32.242+0000 7efef2c76700 1 -- 172.21.15.37:0/3618815452 wait complete. 2024-04-25T21:09:32.243 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:09:32.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:32 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3618815452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:09:33.064 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T21:09:33.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T21:09:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:32 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3618815452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:09:33.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:32 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3618815452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:09:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:09:33.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:33 smithi037 ceph-mon[29195]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T21:09:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:33 smithi177 ceph-mon[33043]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T21:09:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:33 smithi098 ceph-mon[32569]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T21:09:34.949 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:34 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:34 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:35.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:34 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 -- 172.21.15.37:0/1969007556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4101940 msgr2=0x7fc7b4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/1969007556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4101940 0x7fc7b4101dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009990 tx=0x7fc7a0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 -- 172.21.15.37:0/1969007556 shutdown_connections 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/1969007556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4102300 0x7fc7b410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/1969007556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4107930 0x7fc7b4107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/1969007556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4101940 0x7fc7b4101dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 -- 172.21.15.37:0/1969007556 >> 172.21.15.37:0/1969007556 conn(0x7fc7b40788e0 msgr2=0x7fc7b4078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 -- 172.21.15.37:0/1969007556 shutdown_connections 2024-04-25T21:09:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.416+0000 7fc7baaf9700 1 -- 172.21.15.37:0/1969007556 wait complete. 2024-04-25T21:09:35.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.417+0000 7fc7baaf9700 1 Processor -- start 2024-04-25T21:09:35.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.417+0000 7fc7baaf9700 1 -- start start 2024-04-25T21:09:35.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4101940 0x7fc7b4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4107930 0x7fc7b419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4111b90 con 0x7fc7b4102300 2024-04-25T21:09:35.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4111a10 con 0x7fc7b4107930 2024-04-25T21:09:35.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7baaf9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4111d10 con 0x7fc7b4101940 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4101940 0x7fc7b4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.418+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48032/0 (socket says 172.21.15.37:48032) 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 -- 172.21.15.37:0/2003528675 learned_addr learned my addr 172.21.15.37:0/2003528675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b9096700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4107930 0x7fc7b419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 -- 172.21.15.37:0/2003528675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4101940 msgr2=0x7fc7b4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4101940 0x7fc7b4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 -- 172.21.15.37:0/2003528675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4107930 msgr2=0x7fc7b419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4107930 0x7fc7b419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 -- 172.21.15.37:0/2003528675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419cee0 con 0x7fc7b4102300 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000ba30 tx=0x7fc7a000ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.419+0000 7fc7b1ffb700 1 -- 172.21.15.37:0/2003528675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0029070 con 0x7fc7b4102300 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.420+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b419d170 con 0x7fc7b4102300 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.420+0000 7fc7b1ffb700 1 -- 172.21.15.37:0/2003528675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a0003680 con 0x7fc7b4102300 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.420+0000 7fc7b1ffb700 1 -- 172.21.15.37:0/2003528675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0029070 con 0x7fc7b4102300 2024-04-25T21:09:35.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.420+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b406a720 con 0x7fc7b4102300 2024-04-25T21:09:35.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.422+0000 7fc7b1ffb700 1 -- 172.21.15.37:0/2003528675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fc7a002c070 con 0x7fc7b4102300 2024-04-25T21:09:35.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.423+0000 7fc7b1ffb700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 0x7fc7a40a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.423+0000 7fc7b1ffb700 1 -- 172.21.15.37:0/2003528675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc7a0004030 con 0x7fc7b4102300 2024-04-25T21:09:35.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.423+0000 7fc7b8895700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 0x7fc7a40a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.424+0000 7fc7b8895700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 0x7fc7a40a6750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800db20 tx=0x7fc7a800d4b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.425+0000 7fc7c5382b80 1 Processor -- start 2024-04-25T21:09:35.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.425+0000 7fc7c5382b80 1 -- start start 2024-04-25T21:09:35.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.425+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b0f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.425+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.425+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 0x55e56b103660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b0f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b0f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42768/0 (socket says 172.21.15.37:42768) 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b8895700 1 -- 172.21.15.37:0/4255340436 learned_addr learned my addr 172.21.15.37:0/4255340436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:43702/0 (socket says 172.21.15.37:43702) 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e56b048ec0 con 0x55e56b101270 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e56afefaf0 con 0x55e56b0f63a0 2024-04-25T21:09:35.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e56b044e80 con 0x55e56b0f5a80 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 0x55e56b103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4255340436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 msgr2=0x55e56b0f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b0f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4255340436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 msgr2=0x55e56b103660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 0x55e56b103660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4255340436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56af96a60 con 0x55e56b0f63a0 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.426+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 0x55e56b103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:35.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.427+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00d0cd0 tx=0x7fc7a009b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33bf15cb3d6427b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.427+0000 7fc79effd700 1 -- 172.21.15.37:0/4255340436 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a009ef10 con 0x55e56b0f63a0 2024-04-25T21:09:35.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.428+0000 7fc79effd700 1 -- 172.21.15.37:0/4255340436 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a009fcb0 con 0x55e56b0f63a0 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.428+0000 7fc79effd700 1 -- 172.21.15.37:0/4255340436 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00a7f60 con 0x55e56b0f63a0 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 msgr2=0x55e56b100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc7a00d0cd0 tx=0x7fc7a009b8d0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 shutdown_connections 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b101270 0x55e56b103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b0f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4255340436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f63a0 0x55e56b100d30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 >> 172.21.15.37:0/4255340436 conn(0x55e56b023820 msgr2=0x55e56b0f1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 shutdown_connections 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.429+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4255340436 wait complete. 2024-04-25T21:09:35.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.430+0000 7fc7c5382b80 1 -- start start 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.430+0000 7fc7c5382b80 1 Processor -- start 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 0x55e56b100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b101270 0x55e56b0fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e56afefaf0 con 0x55e56b0f5a80 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 0x55e56b100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e56afe7d60 con 0x55e56b101270 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e56b107fb0 con 0x55e56b0f63a0 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42776/0 (socket says 172.21.15.37:42776) 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4089014846 learned_addr learned my addr 172.21.15.37:0/4089014846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:35.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b101270 0x55e56b0fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4089014846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b101270 msgr2=0x55e56b0fe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b101270 0x55e56b0fe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.431+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4089014846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 msgr2=0x55e56b100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7c5382b80 10 client.0 _ll_register_callbacks cb 0x55e56afe1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 0x55e56b100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7b3fff700 1 -- 172.21.15.37:0/4089014846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56b000400 con 0x55e56b0f63a0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a009fcb0 con 0x55e56b0f63a0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7a009fe60 tx=0x7fc7a0019d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56b4beeb8e1b54f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc793fff700 10 client.25126 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc7b8895700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 0x55e56b100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:35.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc793fff700 1 -- 172.21.15.37:0/4089014846 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00bd050 con 0x55e56b0f63a0 2024-04-25T21:09:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc793fff700 1 -- 172.21.15.37:0/4089014846 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc7a00a4d00 con 0x55e56b0f63a0 2024-04-25T21:09:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.432+0000 7fc793fff700 1 -- 172.21.15.37:0/4089014846 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7a00b8060 con 0x55e56b0f63a0 2024-04-25T21:09:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 10 client.25126 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55e56b112240 con 0x55e56b0f63a0 2024-04-25T21:09:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc793fff700 1 -- 172.21.15.37:0/4089014846 <== mon.1 v2:172.21.15.177:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc7a00be240 con 0x55e56b0f63a0 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 10 client.25126 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 10 client.25126 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 20 client.25126 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b114710 0x55e56b116b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 4 client.25126 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e56b0566b0 con 0x55e56b114710 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.433+0000 7fc7b8895700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b114710 0x55e56b116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.434+0000 7fc7b8895700 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b114710 0x55e56b116b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816c2983c2849acf server_cookie=d087d0be471738f5 in_seq=0 out_seq=0 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.434+0000 7fc793fff700 10 client.25126 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.434+0000 7fc793fff700 1 -- 172.21.15.37:0/4089014846 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e56b0566b0 con 0x55e56b114710 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.434+0000 7fc793fff700 10 client.25126 handle_command_reply: tid=0 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.434+0000 7fc7c5382b80 1 client.25126 shutdown 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 msgr2=0x55e56b0fddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7a009fe60 tx=0x7fc7a0019d00 comp rx=0 tx=0).stop 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 shutdown_connections 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b114710 0x55e56b116b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b101270 0x55e56b0fe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f5a80 0x55e56b100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc793fff700 0 client.25126 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4089014846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b0fddd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 >> 172.21.15.37:0/4089014846 conn(0x55e56b023820 msgr2=0x55e56b0f1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 shutdown_connections 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4089014846 wait complete. 2024-04-25T21:09:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.435+0000 7fc7c5382b80 20 client.25126 trim_cache size 0 max 16384 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 Processor -- start 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 -- start start 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f5a80 0x55e56b100610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b10d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41aae10 con 0x55e56b023820 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ac950 con 0x55e56b0f5a80 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.474+0000 7fc7c5382b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ab4e0 con 0x55e56b0f63a0 2024-04-25T21:09:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48056/0 (socket says 172.21.15.37:48056) 2024-04-25T21:09:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b8895700 1 -- 172.21.15.37:0/4043425920 learned_addr learned my addr 172.21.15.37:0/4043425920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 -- 172.21.15.37:0/4043425920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f5a80 msgr2=0x55e56b100610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f5a80 0x55e56b100610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 -- 172.21.15.37:0/4043425920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 msgr2=0x55e56b110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 -- 172.21.15.37:0/4043425920 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e56b0f7200 con 0x55e56b0f63a0 2024-04-25T21:09:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b8895700 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.475+0000 7fc7b9096700 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b10d090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55e56b0f7410 tx=0x7fc7ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6acc1d19520f337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.476+0000 7fc79e7fc700 1 -- 172.21.15.37:0/4043425920 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac004db0 con 0x55e56b0f63a0 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.476+0000 7fc79e7fc700 1 -- 172.21.15.37:0/4043425920 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac01b070 con 0x55e56b0f63a0 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.476+0000 7fc79e7fc700 1 -- 172.21.15.37:0/4043425920 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac01fd30 con 0x55e56b0f63a0 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4043425920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 msgr2=0x55e56b10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b10d090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55e56b0f7410 tx=0x7fc7ac009c00 comp rx=0 tx=0).stop 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4043425920 shutdown_connections 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f63a0 0x55e56b10d090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b023820 0x55e56b110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/4043425920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b0f5a80 0x55e56b100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4043425920 >> 172.21.15.37:0/4043425920 conn(0x55e56b0f21a0 msgr2=0x55e56b110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4043425920 shutdown_connections 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- 172.21.15.37:0/4043425920 wait complete. 2024-04-25T21:09:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 -- start start 2024-04-25T21:09:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.477+0000 7fc7c5382b80 1 Processor -- start 2024-04-25T21:09:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b10c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b10c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b8895700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:43726/0 (socket says 172.21.15.37:43726) 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b9096700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b10c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42800/0 (socket says 172.21.15.37:42800) 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b9096700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48066/0 (socket says 172.21.15.37:48066) 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7b8895700 1 -- 172.21.15.37:0/2213899644 learned_addr learned my addr 172.21.15.37:0/2213899644 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41ac280 con 0x55e56b0f63a0 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41abbb0 con 0x55e56b023820 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.478+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b41a8dd0 con 0x55e56b0f5a80 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 -- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 msgr2=0x55e56b10c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b10c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 -- 172.21.15.37:0/2213899644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 msgr2=0x55e56b10c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e56b17aec0 con 0x55e56b023820 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7c5382b80 10 client.0 _ll_register_callbacks cb 0x55e56afe1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7ac01b070 con 0x55e56b023820 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b9096700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc7b8895700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8028370 tx=0x7fc7a80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57fe365a054d24df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.479+0000 7fc793fff700 10 client.25112 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc793fff700 1 -- 172.21.15.37:0/2213899644 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8011640 con 0x55e56b023820 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc793fff700 1 -- 172.21.15.37:0/2213899644 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc7a80117e0 con 0x55e56b023820 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc793fff700 1 -- 172.21.15.37:0/2213899644 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7a80119e0 con 0x55e56b023820 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 10 client.25112 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55e56b17bff0 con 0x55e56b023820 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc793fff700 1 -- 172.21.15.37:0/2213899644 <== mon.2 v2:172.21.15.98:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc7a8011b80 con 0x55e56b023820 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 10 client.25112 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 10 client.25112 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 20 client.25112 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b101270 0x55e56b17e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 4 client.25112 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T21:09:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.480+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x55e56b050750 con 0x55e56b101270 2024-04-25T21:09:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.481+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b101270 0x55e56b17e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.481+0000 7fc7b3fff700 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b101270 0x55e56b17e730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d59770103c2ab84 server_cookie=ecb4e8087f0b30e5 in_seq=0 out_seq=0 2024-04-25T21:09:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.481+0000 7fc793fff700 10 client.25112 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc793fff700 1 -- 172.21.15.37:0/2213899644 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e56b050750 con 0x55e56b101270 2024-04-25T21:09:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc793fff700 10 client.25112 handle_command_reply: tid=0 2024-04-25T21:09:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 client.25112 shutdown 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 msgr2=0x55e56b10bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8028370 tx=0x7fc7a80047c0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 shutdown_connections 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e56b101270 0x55e56b17e730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc793fff700 0 client.25112 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56b0f63a0 0x55e56b10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e56b023820 0x55e56b10bd50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 --2- 172.21.15.37:0/2213899644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e56b0f5a80 0x55e56b10c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 >> 172.21.15.37:0/2213899644 conn(0x55e56b0f21a0 msgr2=0x55e56b110b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.491+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 shutdown_connections 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.492+0000 7fc7c5382b80 1 -- 172.21.15.37:0/2213899644 wait complete. 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.492+0000 7fc7c5382b80 20 client.25112 trim_cache size 0 max 16384 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "2bb2dd22": { 2024-04-25T21:09:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.2, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:09:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 msgr2=0x7fc7a40a6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 0x7fc7a40a6750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800db20 tx=0x7fc7a800d4b0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 msgr2=0x7fc7b4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000ba30 tx=0x7fc7a000ba60 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 shutdown_connections 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc7a40a4290 0x7fc7a40a6750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7b4107930 0x7fc7b419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7b4101940 0x7fc7b4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 --2- 172.21.15.37:0/2003528675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b4102300 0x7fc7b4198460 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 >> 172.21.15.37:0/2003528675 conn(0x7fc7b40788e0 msgr2=0x7fc7b40ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 shutdown_connections 2024-04-25T21:09:35.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:09:35.493+0000 7fc7baaf9700 1 -- 172.21.15.37:0/2003528675 wait complete. 2024-04-25T21:09:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:35 smithi177 ceph-mon[33043]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T21:09:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:35 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:35 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:36.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:35 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:09:36.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:35 smithi037 ceph-mon[29195]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T21:09:36.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:35 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:36.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:35 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:36.104 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:35 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:09:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:35 smithi098 ceph-mon[32569]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T21:09:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:35 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:09:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:35 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:09:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:35 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:09:36.344 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2bb2dd22', quiesce took 11.2 seconds. Will release after: 26.2 2024-04-25T21:09:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.536+0000 7ff7ffa79640 1 -- 172.21.15.37:0/11465050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f81010f0 msgr2=0x7ff7f81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.536+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/11465050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f81010f0 0x7ff7f81014d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009900 tx=0x7ff7f402a410 comp rx=0 tx=0).stop 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.536+0000 7ff7ffa79640 1 -- 172.21.15.37:0/11465050 shutdown_connections 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.536+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/11465050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f8106170 0x7ff7f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.536+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/11465050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8101aa0 0x7ff7f8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.537+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/11465050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f81010f0 0x7ff7f81014d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.537+0000 7ff7ffa79640 1 -- 172.21.15.37:0/11465050 >> 172.21.15.37:0/11465050 conn(0x7ff7f8077fd0 msgr2=0x7ff7f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.537+0000 7ff7ffa79640 1 -- 172.21.15.37:0/11465050 shutdown_connections 2024-04-25T21:09:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.537+0000 7ff7ffa79640 1 -- 172.21.15.37:0/11465050 wait complete. 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.538+0000 7ff7ffa79640 1 Processor -- start 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.538+0000 7ff7ffa79640 1 -- start start 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8106170 0x7ff7f8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 0x7ff7f819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81110b0 con 0x7ff7f819bf10 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7fe016640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 0x7ff7f819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:43744/0 (socket says 172.21.15.37:43744) 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7fd815640 1 -- 172.21.15.37:0/2669706464 learned_addr learned my addr 172.21.15.37:0/2669706464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7fd014640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8106170 0x7ff7f8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8110f30 con 0x7ff7f81010f0 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.539+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111230 con 0x7ff7f8106170 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 -- 172.21.15.37:0/2669706464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8106170 msgr2=0x7ff7f8197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8106170 0x7ff7f8197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 -- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 msgr2=0x7ff7f819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 0x7ff7f819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 -- 172.21.15.37:0/2669706464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f819ca00 con 0x7ff7f81010f0 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fe016640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 0x7ff7f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.540+0000 7ff7fd815640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005940 tx=0x7ff7f4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.541+0000 7ff7eeffd640 1 -- 172.21.15.37:0/2669706464 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4035050 con 0x7ff7f81010f0 2024-04-25T21:09:36.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.541+0000 7ff7eeffd640 1 -- 172.21.15.37:0/2669706464 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f403bd90 con 0x7ff7f81010f0 2024-04-25T21:09:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.541+0000 7ff7eeffd640 1 -- 172.21.15.37:0/2669706464 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40443e0 con 0x7ff7f81010f0 2024-04-25T21:09:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.541+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f819cc90 con 0x7ff7f81010f0 2024-04-25T21:09:36.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.541+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f80697e0 con 0x7ff7f81010f0 2024-04-25T21:09:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.544+0000 7ff7eeffd640 1 -- 172.21.15.37:0/2669706464 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff7f40035f0 con 0x7ff7f81010f0 2024-04-25T21:09:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.544+0000 7ff80278d740 1 Processor -- start 2024-04-25T21:09:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 -- start start 2024-04-25T21:09:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e346d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5614e33d2970 con 0x5614e346d560 2024-04-25T21:09:36.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5614e336f4a0 con 0x5614e346de80 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff80278d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614e33ce930 con 0x5614e3481010 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff7fe016640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e346d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42832/0 (socket says 172.21.15.37:42832) 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.545+0000 7ff7fd014640 1 -- 172.21.15.37:0/3431170773 learned_addr learned my addr 172.21.15.37:0/3431170773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 -- 172.21.15.37:0/3431170773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 msgr2=0x5614e3480ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 -- 172.21.15.37:0/3431170773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 msgr2=0x5614e346d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e346d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 -- 172.21.15.37:0/3431170773 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e30e5210 con 0x5614e3481010 2024-04-25T21:09:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fe016640 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e346d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800dc90 tx=0x7ff7e800c2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eaffb9a4ca12811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7eeffd640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 0x7ff7e40a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7e3fff640 1 -- 172.21.15.37:0/3431170773 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e801a030 con 0x5614e3481010 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7e3fff640 1 -- 172.21.15.37:0/3431170773 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e8009d10 con 0x5614e3481010 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7e3fff640 1 -- 172.21.15.37:0/3431170773 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8004dd0 con 0x5614e3481010 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.546+0000 7ff7eeffd640 1 -- 172.21.15.37:0/2669706464 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff7f40f54c0 con 0x7ff7f81010f0 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.547+0000 7ff7fe016640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 0x7ff7e40a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff7fe016640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 0x7ff7e40a64b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005fd0 tx=0x7ff7f0009500 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- 172.21.15.37:0/3431170773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 msgr2=0x5614e3483400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800dc90 tx=0x7ff7e800c2f0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- 172.21.15.37:0/3431170773 shutdown_connections 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3483400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 --2- 172.21.15.37:0/3431170773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e346d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- 172.21.15.37:0/3431170773 >> 172.21.15.37:0/3431170773 conn(0x5614e3469b30 msgr2=0x5614e3469f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- 172.21.15.37:0/3431170773 shutdown_connections 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- 172.21.15.37:0/3431170773 wait complete. 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.548+0000 7ff80278d740 1 -- start start 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 Processor -- start 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e3480990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 0x5614e347dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5614e3483be0 con 0x5614e346d560 2024-04-25T21:09:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5614e336d0b0 con 0x5614e3481010 2024-04-25T21:09:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614e3483d60 con 0x5614e346de80 2024-04-25T21:09:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff80278d740 10 client.0 _ll_register_callbacks cb 0x5614e335ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 0x5614e347dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e3480990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 0x5614e347dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42834/0 (socket says 172.21.15.37:42834) 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.549+0000 7ff7fd014640 1 -- 172.21.15.37:0/941177337 learned_addr learned my addr 172.21.15.37:0/941177337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fe016640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 -- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 msgr2=0x5614e347dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 0x5614e347dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 -- 172.21.15.37:0/941177337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 msgr2=0x5614e347e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 -- 172.21.15.37:0/941177337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e31b1620 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7fd815640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e3480990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40f2a00 tx=0x7ff7f40bd5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686ec5425ef7c017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.550+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40c1950 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff7f40c51b0 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e8009d10 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7f40c1250 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff80278d740 10 client.15531 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614e3483be0 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e8009d10 con 0x5614e346d560 2024-04-25T21:09:36.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff7f4043070 con 0x5614e346d560 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff80278d740 10 client.15531 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.551+0000 7ff80278d740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff80278d740 20 client.15531 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e3490e60 0x5614e3493250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff80278d740 4 client.15531 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e33e0390 con 0x5614e3490e60 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff7fd014640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e3490e60 0x5614e3493250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff7fd014640 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e3490e60 0x5614e3493250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba7c3c837b15133 server_cookie=461f742fdd8d32c1 in_seq=0 out_seq=0 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.552+0000 7ff7e17fa640 10 client.15531 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.553+0000 7ff7e17fa640 1 -- 172.21.15.37:0/941177337 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614e33e0390 con 0x5614e3490e60 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.553+0000 7ff7e17fa640 10 client.15531 handle_command_reply: tid=0 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.553+0000 7ff80278d740 1 client.15531 shutdown 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.553+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 msgr2=0x5614e3480990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.553+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e3480990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40f2a00 tx=0x7ff7f40bd5b0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 shutdown_connections 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e3490e60 0x5614e3493250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff7e17fa640 0 client.15531 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346de80 0x5614e347dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 --2- 172.21.15.37:0/941177337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e3480990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 >> 172.21.15.37:0/941177337 conn(0x5614e3469b30 msgr2=0x5614e346f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 shutdown_connections 2024-04-25T21:09:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 1 -- 172.21.15.37:0/941177337 wait complete. 2024-04-25T21:09:36.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.554+0000 7ff80278d740 20 client.15531 trim_cache size 0 max 16384 2024-04-25T21:09:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 Processor -- start 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 -- start start 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e348cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3498af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81ab7d0 con 0x5614e346d560 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81ad310 con 0x5614e346de80 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.584+0000 7ff80278d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81abea0 con 0x5614e3481010 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e348cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fe016640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3498af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fe016640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3498af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42848/0 (socket says 172.21.15.37:42848) 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fe016640 1 -- 172.21.15.37:0/3913699008 learned_addr learned my addr 172.21.15.37:0/3913699008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 -- 172.21.15.37:0/3913699008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 msgr2=0x5614e3498af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3498af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 -- 172.21.15.37:0/3913699008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 msgr2=0x5614e348cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e348cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd014640 1 -- 172.21.15.37:0/3913699008 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e346a8c0 con 0x5614e346de80 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.585+0000 7ff7fd815640 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e348cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.586+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5614e3493a60 tx=0x7ff7e8004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e73324eb00b0ac38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.586+0000 7ff7e37fe640 1 -- 172.21.15.37:0/3913699008 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e801a050 con 0x5614e346de80 2024-04-25T21:09:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.586+0000 7ff7e37fe640 1 -- 172.21.15.37:0/3913699008 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80044f0 con 0x5614e346de80 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.586+0000 7ff7e37fe640 1 -- 172.21.15.37:0/3913699008 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003d50 con 0x5614e346de80 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 -- 172.21.15.37:0/3913699008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 msgr2=0x5614e3480530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5614e3493a60 tx=0x7ff7e8004970 comp rx=0 tx=0).stop 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 -- 172.21.15.37:0/3913699008 shutdown_connections 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e3481010 0x5614e3498af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e346de80 0x5614e3480530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 --2- 172.21.15.37:0/3913699008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346d560 0x5614e348cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.587+0000 7ff80278d740 1 -- 172.21.15.37:0/3913699008 >> 172.21.15.37:0/3913699008 conn(0x5614e3469b30 msgr2=0x5614e3479300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.588+0000 7ff80278d740 1 -- 172.21.15.37:0/3913699008 shutdown_connections 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.588+0000 7ff80278d740 1 -- 172.21.15.37:0/3913699008 wait complete. 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.588+0000 7ff80278d740 1 -- start start 2024-04-25T21:09:36.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.588+0000 7ff80278d740 1 Processor -- start 2024-04-25T21:09:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 0x5614e347ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81acc40 con 0x5614e346de80 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81ac570 con 0x5614e3481010 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff7fd014640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 0x5614e347ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff7fd815640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:42860/0 (socket says 172.21.15.37:42860) 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff7fd815640 1 -- 172.21.15.37:0/3393329315 learned_addr learned my addr 172.21.15.37:0/3393329315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff7fe016640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81a97b0 con 0x5614e346d560 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.589+0000 7ff80278d740 10 client.0 _ll_register_callbacks cb 0x7ff7f40095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 -- 172.21.15.37:0/3393329315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 msgr2=0x5614e347b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 -- 172.21.15.37:0/3393329315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 msgr2=0x5614e347ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 0x5614e347ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 -- 172.21.15.37:0/3393329315 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e34f6360 con 0x5614e346d560 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 0x5614e347ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7fd815640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40f5490 tx=0x7ff7f40bd620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a5216cefd9e76d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40c1950 con 0x5614e346d560 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff7f40c06f0 con 0x5614e346d560 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e80044f0 con 0x5614e346d560 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7f40c5600 con 0x5614e346d560 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff80278d740 10 client.25150 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.590+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614e34f6360 con 0x5614e346d560 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.591+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f40c9ec0 con 0x5614e346d560 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff7f4043070 con 0x5614e346d560 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 10 client.25150 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 10 client.25150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 20 client.25150 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e34f7950 0x5614e34f9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 4 client.25150 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff7e80044f0 con 0x5614e34f7950 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.592+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e34f7950 0x5614e34f9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.593+0000 7ff7e17fa640 10 client.25150 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.593+0000 7ff7fd014640 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e34f7950 0x5614e34f9d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbac3a8481be1935 server_cookie=e71976bd5522c8d1 in_seq=0 out_seq=0 2024-04-25T21:09:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.600+0000 7ff7e17fa640 1 -- 172.21.15.37:0/3393329315 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff7e80044f0 con 0x5614e34f7950 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.600+0000 7ff7e17fa640 10 client.25150 handle_command_reply: tid=0 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.600+0000 7ff80278d740 1 client.25150 shutdown 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 msgr2=0x5614e347a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40f5490 tx=0x7ff7f40bd620 comp rx=0 tx=0).stop 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 shutdown_connections 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614e34f7950 0x5614e34f9d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614e3481010 0x5614e347b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614e346de80 0x5614e347ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 --2- 172.21.15.37:0/3393329315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614e346d560 0x5614e347a940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff7e17fa640 0 client.25150 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:36.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 >> 172.21.15.37:0/3393329315 conn(0x5614e3469b30 msgr2=0x5614e346f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 shutdown_connections 2024-04-25T21:09:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 1 -- 172.21.15.37:0/3393329315 wait complete. 2024-04-25T21:09:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.601+0000 7ff80278d740 20 client.25150 trim_cache size 0 max 16384 2024-04-25T21:09:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:09:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:09:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:09:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:09:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 msgr2=0x7ff7e40a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 0x7ff7e40a64b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005fd0 tx=0x7ff7f0009500 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 msgr2=0x7ff7f8197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4005940 tx=0x7ff7f4004270 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 shutdown_connections 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7f819bf10 0x7ff7f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff7e40a3ff0 0x7ff7e40a64b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7f8106170 0x7ff7f8197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 --2- 172.21.15.37:0/2669706464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7f81010f0 0x7ff7f8197650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 >> 172.21.15.37:0/2669706464 conn(0x7ff7f8077fd0 msgr2=0x7ff7f80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 shutdown_connections 2024-04-25T21:09:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:36.602+0000 7ff7ffa79640 1 -- 172.21.15.37:0/2669706464 wait complete. 2024-04-25T21:09:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:37 smithi177 ceph-mon[33043]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T21:09:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:37 smithi098 ceph-mon[32569]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T21:09:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:37 smithi037 ceph-mon[29195]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T21:09:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:39 smithi177 ceph-mon[33043]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T21:09:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:39 smithi098 ceph-mon[32569]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T21:09:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:39 smithi037 ceph-mon[29195]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-25T21:09:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:41 smithi177 ceph-mon[33043]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T21:09:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:41 smithi098 ceph-mon[32569]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T21:09:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:41 smithi037 ceph-mon[29195]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-25T21:09:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:43 smithi177 ceph-mon[33043]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:43 smithi098 ceph-mon[32569]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:44.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:43 smithi037 ceph-mon[29195]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:45 smithi177 ceph-mon[33043]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:45 smithi098 ceph-mon[32569]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:46.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:45 smithi037 ceph-mon[29195]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:47.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:46 smithi177 ceph-mon[33043]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:47.128 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:46 smithi037 ceph-mon[29195]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:47.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:46 smithi098 ceph-mon[32569]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-25T21:09:49.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:49 smithi037 ceph-mon[29195]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:49.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:49 smithi177 ceph-mon[33043]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:49.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:49 smithi098 ceph-mon[32569]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:09:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 -- 172.21.15.37:0/907135483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe098106170 msgr2=0x7fe09810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/907135483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe098106170 0x7fe09810dd40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ca0 tx=0x7fe09002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 -- 172.21.15.37:0/907135483 shutdown_connections 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/907135483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe098106170 0x7fe09810dd40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/907135483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe098101aa0 0x7fe098105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/907135483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0981010f0 0x7fe0981014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.720+0000 7fe09f8a7640 1 -- 172.21.15.37:0/907135483 >> 172.21.15.37:0/907135483 conn(0x7fe098077fd0 msgr2=0x7fe0980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.721+0000 7fe09f8a7640 1 -- 172.21.15.37:0/907135483 shutdown_connections 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.721+0000 7fe09f8a7640 1 -- 172.21.15.37:0/907135483 wait complete. 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.721+0000 7fe09f8a7640 1 Processor -- start 2024-04-25T21:09:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.721+0000 7fe09f8a7640 1 -- start start 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0981010f0 0x7fe0981975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 0x7fe098197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981110e0 con 0x7fe09819be80 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe098110f60 con 0x7fe098106170 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09f8a7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111260 con 0x7fe0981010f0 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46080/0 (socket says 172.21.15.37:46080) 2024-04-25T21:09:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 -- 172.21.15.37:0/753141554 learned_addr learned my addr 172.21.15.37:0/753141554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09d643640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0981010f0 0x7fe0981975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09ce42640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 0x7fe098197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 -- 172.21.15.37:0/753141554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0981010f0 msgr2=0x7fe0981975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0981010f0 0x7fe0981975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 -- 172.21.15.37:0/753141554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 msgr2=0x7fe098197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 0x7fe098197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09de44640 1 -- 172.21.15.37:0/753141554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819c970 con 0x7fe09819be80 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.722+0000 7fe09ce42640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 0x7fe098197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe09de44640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe0900098d0 tx=0x7fe0900317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe08e7fc640 1 -- 172.21.15.37:0/753141554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090003b70 con 0x7fe09819be80 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09819cc00 con 0x7fe09819be80 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe08e7fc640 1 -- 172.21.15.37:0/753141554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe090003d10 con 0x7fe09819be80 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe08e7fc640 1 -- 172.21.15.37:0/753141554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09002fa20 con 0x7fe09819be80 2024-04-25T21:09:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.723+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0980698d0 con 0x7fe09819be80 2024-04-25T21:09:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.726+0000 7fe08e7fc640 1 -- 172.21.15.37:0/753141554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe09002fcc0 con 0x7fe09819be80 2024-04-25T21:09:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.726+0000 7fe0a25bf740 1 Processor -- start 2024-04-25T21:09:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.726+0000 7fe0a25bf740 1 -- start start 2024-04-25T21:09:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f68b620 0x56276f68ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 0x56276f698bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56276f5f0970 con 0x56276f68bf40 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56276f58d4a0 con 0x56276f68b620 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56276f5ec930 con 0x56276f696800 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46094/0 (socket says 172.21.15.37:46094) 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 -- 172.21.15.37:0/298003637 learned_addr learned my addr 172.21.15.37:0/298003637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09ce42640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f68b620 0x56276f68ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe08e7fc640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 0x7fe0840aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09ce42640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 0x56276f698bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 -- 172.21.15.37:0/298003637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 msgr2=0x56276f698bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09de44640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 0x7fe0840aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 0x56276f698bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 -- 172.21.15.37:0/298003637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f68b620 msgr2=0x56276f68ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f68b620 0x56276f68ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 -- 172.21.15.37:0/298003637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276f303210 con 0x56276f68bf40 2024-04-25T21:09:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09ce42640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 0x56276f698bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.727+0000 7fe09d643640 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe094009c70 tx=0x7fe094005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd7053ed402d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.728+0000 7fe08e7fc640 1 -- 172.21.15.37:0/753141554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe0900f1e80 con 0x7fe09819be80 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.728+0000 7fe0837fe640 1 -- 172.21.15.37:0/298003637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094004030 con 0x56276f68bf40 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.728+0000 7fe0837fe640 1 -- 172.21.15.37:0/298003637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0940041d0 con 0x56276f68bf40 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.728+0000 7fe0837fe640 1 -- 172.21.15.37:0/298003637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0940177a0 con 0x56276f68bf40 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- 172.21.15.37:0/298003637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 msgr2=0x56276f6962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe09de44640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 0x7fe0840aa840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0900ef530 tx=0x7fe0900b99c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe094009c70 tx=0x7fe094005f90 comp rx=0 tx=0).stop 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- 172.21.15.37:0/298003637 shutdown_connections 2024-04-25T21:09:49.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f696800 0x56276f698bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f6962c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/298003637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f68b620 0x56276f68ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- 172.21.15.37:0/298003637 >> 172.21.15.37:0/298003637 conn(0x56276f687b30 msgr2=0x56276f687f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- 172.21.15.37:0/298003637 shutdown_connections 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- 172.21.15.37:0/298003637 wait complete. 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 -- start start 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.729+0000 7fe0a25bf740 1 Processor -- start 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f696170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56276f699430 con 0x56276f68bf40 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56276f58b0b0 con 0x56276f696800 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56276f6995b0 con 0x56276f68b620 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09d643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09d643640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46098/0 (socket says 172.21.15.37:46098) 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09d643640 1 -- 172.21.15.37:0/1373242960 learned_addr learned my addr 172.21.15.37:0/1373242960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09ce42640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f696170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09ce42640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09d643640 1 -- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 msgr2=0x56276f696170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.730+0000 7fe09d643640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f696170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe09d643640 1 -- 172.21.15.37:0/1373242960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 msgr2=0x56276f6939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe09d643640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe09d643640 1 -- 172.21.15.37:0/1373242960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276f3cf620 con 0x56276f68bf40 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe0a25bf740 10 client.0 _ll_register_callbacks cb 0x56276f578d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe09d643640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe094005bc0 tx=0x7fe0940096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447520e739b30497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe080ff9640 1 -- 172.21.15.37:0/1373242960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094005040 con 0x56276f68bf40 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0940041d0 con 0x56276f68bf40 2024-04-25T21:09:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe080ff9640 1 -- 172.21.15.37:0/1373242960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe0940051e0 con 0x56276f68bf40 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe080ff9640 1 -- 172.21.15.37:0/1373242960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0940053e0 con 0x56276f68bf40 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe0a25bf740 10 client.15549 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.731+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=27}) -- 0x56276f6a5160 con 0x56276f68bf40 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe080ff9640 1 -- 172.21.15.37:0/1373242960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe0940046c0 con 0x56276f68bf40 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 10 client.15549 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 20 client.15549 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f6a64b0 0x56276f6a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 4 client.15549 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:09:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56276f5fe390 con 0x56276f6a64b0 2024-04-25T21:09:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.732+0000 7fe09ce42640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f6a64b0 0x56276f6a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe080ff9640 10 client.15549 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe09ce42640 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f6a64b0 0x56276f6a88a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb43031aebf88414 server_cookie=840890670c60c5c0 in_seq=0 out_seq=0 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe080ff9640 1 -- 172.21.15.37:0/1373242960 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56276f5fe390 con 0x56276f6a64b0 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe080ff9640 10 client.15549 handle_command_reply: tid=0 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe0a25bf740 1 client.15549 shutdown 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.733+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 msgr2=0x56276f693460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe094005bc0 tx=0x7fe0940096e0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 shutdown_connections 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f6a64b0 0x56276f6a88a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68bf40 0x56276f693460 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/1373242960 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f696170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 >> 172.21.15.37:0/1373242960 conn(0x56276f687b30 msgr2=0x56276f68d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe080ff9640 0 client.15549 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 shutdown_connections 2024-04-25T21:09:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 1 -- 172.21.15.37:0/1373242960 wait complete. 2024-04-25T21:09:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.734+0000 7fe0a25bf740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T21:09:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 Processor -- start 2024-04-25T21:09:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 -- start start 2024-04-25T21:09:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f6a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ab850 con 0x56276f692840 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ad390 con 0x56276f696800 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981abf20 con 0x56276f68b620 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe09ce42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.749+0000 7fe09ce42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46110/0 (socket says 172.21.15.37:46110) 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 -- 172.21.15.37:0/950075176 learned_addr learned my addr 172.21.15.37:0/950075176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09d643640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f6a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 -- 172.21.15.37:0/950075176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 msgr2=0x56276f6a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f6a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 -- 172.21.15.37:0/950075176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 msgr2=0x56276f6adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 -- 172.21.15.37:0/950075176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56276f6832e0 con 0x56276f692840 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09d643640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe09ce42640 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe0880058c0 tx=0x7fe088015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f718ad5a8ddb3b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe082ffd640 1 -- 172.21.15.37:0/950075176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088021410 con 0x56276f692840 2024-04-25T21:09:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe082ffd640 1 -- 172.21.15.37:0/950075176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe088011780 con 0x56276f692840 2024-04-25T21:09:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.750+0000 7fe082ffd640 1 -- 172.21.15.37:0/950075176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880103d0 con 0x56276f692840 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 -- 172.21.15.37:0/950075176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 msgr2=0x56276f6ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe0880058c0 tx=0x7fe088015040 comp rx=0 tx=0).stop 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 -- 172.21.15.37:0/950075176 shutdown_connections 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f692840 0x56276f6ae3e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f6adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/950075176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f68b620 0x56276f6a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 -- 172.21.15.37:0/950075176 >> 172.21.15.37:0/950075176 conn(0x56276f687b30 msgr2=0x56276f686cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 -- 172.21.15.37:0/950075176 shutdown_connections 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.751+0000 7fe0a25bf740 1 -- 172.21.15.37:0/950075176 wait complete. 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 -- start start 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 Processor -- start 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 0x56276f69a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f69a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981accc0 con 0x56276f68b620 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ac5f0 con 0x56276f696800 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981a9830 con 0x56276f692840 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe0a25bf740 10 client.0 _ll_register_callbacks cb 0x7fe094003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe09ce42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe09ce42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46118/0 (socket says 172.21.15.37:46118) 2024-04-25T21:09:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe09ce42640 1 -- 172.21.15.37:0/2537779456 learned_addr learned my addr 172.21.15.37:0/2537779456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.752+0000 7fe09d643640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 0x56276f69a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f69a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 -- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 msgr2=0x56276f69a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 0x56276f69a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 -- 172.21.15.37:0/2537779456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 msgr2=0x56276f69a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f69a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 -- 172.21.15.37:0/2537779456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56276f68bd70 con 0x56276f68b620 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09d643640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 0x56276f69a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe09ce42640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe088000c00 tx=0x7fe088005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5545d46ae1e385ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088011b40 con 0x56276f68b620 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fe088028050 con 0x56276f68b620 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe088011780 con 0x56276f68b620 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe08803f050 con 0x56276f68b620 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe0a25bf740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-04-25T21:09:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.753+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56276f70c6f0 con 0x56276f68b620 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe088011780 con 0x56276f68b620 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe088021ce0 con 0x56276f68b620 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 20 client.15555 populate_metadata read hostname 'smithi037' 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f70ccb0 0x56276f70f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 4 client.15555 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56276f5f8200 con 0x56276f70ccb0 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe09d643640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f70ccb0 0x56276f70f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe09d643640 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f70ccb0 0x56276f70f0a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ae8613ff105a37 server_cookie=fc2f5b19215236b0 in_seq=0 out_seq=0 2024-04-25T21:09:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.754+0000 7fe080ff9640 10 client.15555 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe080ff9640 1 -- 172.21.15.37:0/2537779456 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56276f5f8200 con 0x56276f70ccb0 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe080ff9640 10 client.15555 handle_command_reply: tid=0 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 client.15555 shutdown 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 msgr2=0x56276f69ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe088000c00 tx=0x7fe088005e50 comp rx=0 tx=0).stop 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 shutdown_connections 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56276f70ccb0 0x56276f70f0a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56276f696800 0x56276f69a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56276f692840 0x56276f69a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe080ff9640 0 client.15555 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.758+0000 7fe0a25bf740 1 --2- 172.21.15.37:0/2537779456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56276f68b620 0x56276f69ed40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 >> 172.21.15.37:0/2537779456 conn(0x56276f687b30 msgr2=0x56276f6a12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 shutdown_connections 2024-04-25T21:09:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe0a25bf740 1 -- 172.21.15.37:0/2537779456 wait complete. 2024-04-25T21:09:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe0a25bf740 20 client.15555 trim_cache size 0 max 16384 2024-04-25T21:09:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:09:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:09:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:09:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 msgr2=0x7fe0840aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 0x7fe0840aa840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0900ef530 tx=0x7fe0900b99c0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 msgr2=0x7fe09819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.759+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe0900098d0 tx=0x7fe0900317b0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 shutdown_connections 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe09819be80 0x7fe09819c330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe098106170 0x7fe098197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe0840a8380 0x7fe0840aa840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 --2- 172.21.15.37:0/753141554 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0981010f0 0x7fe0981975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 >> 172.21.15.37:0/753141554 conn(0x7fe098077fd0 msgr2=0x7fe0980fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 shutdown_connections 2024-04-25T21:09:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:09:49.760+0000 7fe09f8a7640 1 -- 172.21.15.37:0/753141554 wait complete. 2024-04-25T21:09:51.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:51 smithi037 ceph-mon[29195]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:51 smithi177 ceph-mon[33043]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:51 smithi098 ceph-mon[32569]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:09:53.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:53 smithi037 ceph-mon[29195]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:09:53.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:53 smithi177 ceph-mon[33043]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:09:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:53 smithi098 ceph-mon[32569]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:09:55.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:55 smithi037 ceph-mon[29195]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-25T21:09:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:55 smithi177 ceph-mon[33043]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-25T21:09:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:55 smithi098 ceph-mon[32569]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-25T21:09:57.333 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to 'e221b808d285b9d02e54cc10c53c84188e8e41e3'. 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-04-25T21:09:57.334 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.335 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-04-25T21:09:57.335 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.335 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T21:09:57.335 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-25T21:09:57.335 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at e221b808d28 qa/suites/fs: add quiescer to the fs suite 2024-04-25T21:09:57.336 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T21:09:57.402 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T21:09:57.404 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T21:09:57.404 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T21:09:57.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:57 smithi037 ceph-mon[29195]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T21:09:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:57 smithi177 ceph-mon[33043]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T21:09:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:57 smithi098 ceph-mon[32569]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-25T21:09:58.003 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T21:09:58.058 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T21:09:58.099 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T21:09:58.101 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T21:09:58.101 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T21:09:58.140 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T21:09:58.144 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T21:09:58.144 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T21:09:58.200 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-25T21:09:58.201 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-25T21:09:58.201 DEBUG:teuthology.orchestra.run.smithi037:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e221b808d285b9d02e54cc10c53c84188e8e41e3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T21:09:59.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:09:59 smithi037 ceph-mon[29195]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:09:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:09:59 smithi177 ceph-mon[33043]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:09:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:09:59 smithi098 ceph-mon[32569]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:00.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:00 smithi037 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T21:10:00.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:00 smithi177 ceph-mon[33043]: overall HEALTH_OK 2024-04-25T21:10:00.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:00 smithi098 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T21:10:01.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:01 smithi037 ceph-mon[29195]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-25T21:10:01.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:01 smithi037 ceph-mon[29195]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T21:10:01.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:01 smithi177 ceph-mon[33043]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-25T21:10:01.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:01 smithi177 ceph-mon[33043]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T21:10:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:01 smithi098 ceph-mon[32569]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-25T21:10:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:01 smithi098 ceph-mon[32569]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T21:10:02.545 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:10:02.814 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 -- 172.21.15.37:0/4248490649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e8106170 msgr2=0x7f64e810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 --2- 172.21.15.37:0/4248490649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e8106170 0x7f64e810dd40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009ca0 tx=0x7f64e002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 -- 172.21.15.37:0/4248490649 shutdown_connections 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 --2- 172.21.15.37:0/4248490649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e8106170 0x7f64e810dd40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 --2- 172.21.15.37:0/4248490649 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e8101aa0 0x7f64e8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 --2- 172.21.15.37:0/4248490649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e81010f0 0x7f64e81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.866+0000 7f64ecd08640 1 -- 172.21.15.37:0/4248490649 >> 172.21.15.37:0/4248490649 conn(0x7f64e8077fd0 msgr2=0x7f64e80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.867+0000 7f64ecd08640 1 -- 172.21.15.37:0/4248490649 shutdown_connections 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.867+0000 7f64ecd08640 1 -- 172.21.15.37:0/4248490649 wait complete. 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.867+0000 7f64ecd08640 1 Processor -- start 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.867+0000 7f64ecd08640 1 -- start start 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.867+0000 7f64ecd08640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e81010f0 0x7f64e810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e8106170 0x7f64e810ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81121f0 con 0x7f64e810f150 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8112070 con 0x7f64e81010f0 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64e8112370 con 0x7f64e8106170 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40564/0 (socket says 172.21.15.37:40564) 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 -- 172.21.15.37:0/1316336524 learned_addr learned my addr 172.21.15.37:0/1316336524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e659c640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e81010f0 0x7f64e810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e8106170 0x7f64e810ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 -- 172.21.15.37:0/1316336524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e8106170 msgr2=0x7f64e810ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e8106170 0x7f64e810ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 -- 172.21.15.37:0/1316336524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e81010f0 msgr2=0x7f64e810e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e81010f0 0x7f64e810e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 -- 172.21.15.37:0/1316336524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e81a6f20 con 0x7f64e810f150 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64e6d9d640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005cc0 tx=0x7f64e002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64d77fe640 1 -- 172.21.15.37:0/1316336524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003bf0 con 0x7f64e810f150 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e81a71b0 con 0x7f64e810f150 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64d77fe640 1 -- 172.21.15.37:0/1316336524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e0003d90 con 0x7f64e810f150 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64d77fe640 1 -- 172.21.15.37:0/1316336524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e002fbc0 con 0x7f64e810f150 2024-04-25T21:10:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.868+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e81a7750 con 0x7f64e810f150 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.871+0000 7f64d77fe640 1 -- 172.21.15.37:0/1316336524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64e002fd60 con 0x7f64e810f150 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.871+0000 7f64efa1d740 1 Processor -- start 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 -- start start 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b38a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b43800 0x55c758b45bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c758a9d970 con 0x55c758b38f40 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c758a3a4a0 con 0x55c758b43800 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64efa1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c758a99930 con 0x55c758b38620 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64d77fe640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 0x7f64d00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40574/0 (socket says 172.21.15.37:40574) 2024-04-25T21:10:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 -- 172.21.15.37:0/3322538469 learned_addr learned my addr 172.21.15.37:0/3322538469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e6d9d640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 0x7f64d00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b38a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 -- 172.21.15.37:0/3322538469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 msgr2=0x55c758b38a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b38a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 -- 172.21.15.37:0/3322538469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b43800 msgr2=0x55c758b45bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b43800 0x55c758b45bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e659c640 1 -- 172.21.15.37:0/3322538469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7587b0210 con 0x55c758b38f40 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b38a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.872+0000 7f64d77fe640 1 -- 172.21.15.37:0/1316336524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f64e00f1b40 con 0x7f64e810f150 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64e659c640 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f64d800bcd0 tx=0x7f64d80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbca33d200fae2a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64e6d9d640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 0x7f64d00a62b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64e00bdbb0 tx=0x7f64e00040e0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64d4ff9640 1 -- 172.21.15.37:0/3322538469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8005220 con 0x55c758b38f40 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64d4ff9640 1 -- 172.21.15.37:0/3322538469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d8004c50 con 0x55c758b38f40 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64d4ff9640 1 -- 172.21.15.37:0/3322538469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8011680 con 0x55c758b38f40 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 -- 172.21.15.37:0/3322538469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 msgr2=0x55c758b432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f64d800bcd0 tx=0x7f64d80107a0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 -- 172.21.15.37:0/3322538469 shutdown_connections 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b43800 0x55c758b45bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b38f40 0x55c758b432c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3322538469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b38a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.873+0000 7f64efa1d740 1 -- 172.21.15.37:0/3322538469 >> 172.21.15.37:0/3322538469 conn(0x55c758b34b30 msgr2=0x55c758b34f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- 172.21.15.37:0/3322538469 shutdown_connections 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- 172.21.15.37:0/3322538469 wait complete. 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- start start 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 Processor -- start 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38620 0x55c758b43190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38f40 0x55c758b40480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c758b46430 con 0x55c758b43800 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c758a380b0 con 0x55c758b38620 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c758b465b0 con 0x55c758b38f40 2024-04-25T21:10:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64efa1d740 10 client.0 _ll_register_callbacks cb 0x55c758a25d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40582/0 (socket says 172.21.15.37:40582) 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3904306083 learned_addr learned my addr 172.21.15.37:0/3904306083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.874+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38620 0x55c758b43190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e659c640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38f40 0x55c758b40480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38f40 msgr2=0x55c758b40480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38f40 0x55c758b40480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3904306083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38620 msgr2=0x55c758b43190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38620 0x55c758b43190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3904306083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c75887c620 con 0x55c758b43800 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64dc013bb0 tx=0x7f64dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738499663ed589de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc011c70 con 0x55c758b43800 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f64dc011e10 con 0x55c758b43800 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d8004c50 con 0x55c758b43800 2024-04-25T21:10:02.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64dc004030 con 0x55c758b43800 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64efa1d740 10 client.15567 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c758b52350 con 0x55c758b43800 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.875+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d8004c50 con 0x55c758b43800 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f64dc003c90 con 0x55c758b43800 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 20 client.15567 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758b537c0 0x55c758b55bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 4 client.15567 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c758aab390 con 0x55c758b537c0 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64e659c640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758b537c0 0x55c758b55bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64cdffb640 10 client.15567 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.876+0000 7f64e659c640 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758b537c0 0x55c758b55bb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9147967120afb03a server_cookie=f5650688218f4929 in_seq=0 out_seq=0 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64cdffb640 1 -- 172.21.15.37:0/3904306083 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c758aab390 con 0x55c758b537c0 2024-04-25T21:10:02.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64cdffb640 10 client.15567 handle_command_reply: tid=0 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 client.15567 shutdown 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 msgr2=0x55c758b40a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64dc013bb0 tx=0x7f64dc0046c0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 shutdown_connections 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758b537c0 0x55c758b55bb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b40a00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38f40 0x55c758b40480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3904306083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38620 0x55c758b43190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 >> 172.21.15.37:0/3904306083 conn(0x55c758b34b30 msgr2=0x55c758b3a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64cdffb640 0 client.15567 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 shutdown_connections 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 1 -- 172.21.15.37:0/3904306083 wait complete. 2024-04-25T21:10:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.877+0000 7f64efa1d740 20 client.15567 trim_cache size 0 max 16384 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 Processor -- start 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 -- start start 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b5b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81afcb0 con 0x55c758b43800 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b17f0 con 0x55c758b38f40 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.892+0000 7f64efa1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b0380 con 0x55c758b38620 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40588/0 (socket says 172.21.15.37:40588) 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3277977398 learned_addr learned my addr 172.21.15.37:0/3277977398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3277977398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 msgr2=0x55c758b42cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b42cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e659c640 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b5b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3277977398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 msgr2=0x55c758b5b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b5b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 -- 172.21.15.37:0/3277977398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c758b45f60 con 0x55c758b43800 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55c758b39360 tx=0x7f64dc013890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fc753a1ef9fecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64cffff640 1 -- 172.21.15.37:0/3277977398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc02f070 con 0x55c758b43800 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64cffff640 1 -- 172.21.15.37:0/3277977398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc017040 con 0x55c758b43800 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.893+0000 7f64cffff640 1 -- 172.21.15.37:0/3277977398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc028d70 con 0x55c758b43800 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- 172.21.15.37:0/3277977398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 msgr2=0x55c758b5b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55c758b39360 tx=0x7f64dc013890 comp rx=0 tx=0).stop 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- 172.21.15.37:0/3277977398 shutdown_connections 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b5b960 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b5b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- 172.21.15.37:0/3277977398 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b42cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- 172.21.15.37:0/3277977398 >> 172.21.15.37:0/3277977398 conn(0x55c758b34b30 msgr2=0x55c758b40440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- 172.21.15.37:0/3277977398 shutdown_connections 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- 172.21.15.37:0/3277977398 wait complete. 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- start start 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 Processor -- start 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b4b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b4bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b1120 con 0x55c758b43800 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81b0a50 con 0x55c758b38f40 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f64e81adc90 con 0x55c758b38620 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64efa1d740 10 client.0 _ll_register_callbacks cb 0x7f64dc0137c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64e5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40596/0 (socket says 172.21.15.37:40596) 2024-04-25T21:10:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.894+0000 7f64e5d9b640 1 -- 172.21.15.37:0/1190952798 learned_addr learned my addr 172.21.15.37:0/1190952798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e659c640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b4bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b4b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 -- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 msgr2=0x55c758b4b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b4b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 -- 172.21.15.37:0/1190952798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 msgr2=0x55c758b4bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b4bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 -- 172.21.15.37:0/1190952798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c758b49500 con 0x55c758b43800 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64e5d9b640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c758b49d80 tx=0x7f64dc011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281e38498f8a8e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc02f040 con 0x55c758b43800 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f64dc0057f0 con 0x55c758b43800 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64dc017040 con 0x55c758b43800 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f64dc005220 con 0x55c758b43800 2024-04-25T21:10:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64efa1d740 10 client.15573 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c758bb98b0 con 0x55c758b43800 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.895+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc017040 con 0x55c758b43800 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f64dc020b10 con 0x55c758b43800 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 20 client.15573 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758bb9f60 0x55c758bbc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 4 client.15573 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c758aa5200 con 0x55c758bb9f60 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64e659c640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758bb9f60 0x55c758bbc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64cdffb640 10 client.15573 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.896+0000 7f64e659c640 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758bb9f60 0x55c758bbc350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22cec98560e5f672 server_cookie=9b9c06a752840e14 in_seq=0 out_seq=0 2024-04-25T21:10:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64cdffb640 1 -- 172.21.15.37:0/1190952798 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c758aa5200 con 0x55c758bb9f60 2024-04-25T21:10:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64cdffb640 10 client.15573 handle_command_reply: tid=0 2024-04-25T21:10:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 client.15573 shutdown 2024-04-25T21:10:02.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 msgr2=0x55c758b48cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55c758b49d80 tx=0x7f64dc011d50 comp rx=0 tx=0).stop 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 shutdown_connections 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c758bb9f60 0x55c758bbc350 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c758b43800 0x55c758b48cd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c758b38f40 0x55c758b4bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64cdffb640 0 client.15573 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 --2- 172.21.15.37:0/1190952798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c758b38620 0x55c758b4b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 >> 172.21.15.37:0/1190952798 conn(0x55c758b34b30 msgr2=0x55c758b33420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 shutdown_connections 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 1 -- 172.21.15.37:0/1190952798 wait complete. 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64efa1d740 20 client.15573 trim_cache size 0 max 16384 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:10:02.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:10:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:10:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:10:02.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 msgr2=0x7f64d00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 0x7f64d00a62b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64e00bdbb0 tx=0x7f64e00040e0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 msgr2=0x7f64e81a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.903+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64e0005cc0 tx=0x7f64e002f380 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 shutdown_connections 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64e810f150 0x7f64e81a6820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f64d00a3df0 0x7f64d00a62b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64e8106170 0x7f64e810ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 --2- 172.21.15.37:0/1316336524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e81010f0 0x7f64e810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 >> 172.21.15.37:0/1316336524 conn(0x7f64e8077fd0 msgr2=0x7f64e80fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 shutdown_connections 2024-04-25T21:10:02.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:02.904+0000 7f64ecd08640 1 -- 172.21.15.37:0/1316336524 wait complete. 2024-04-25T21:10:03.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:03 smithi037 ceph-mon[29195]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:10:03.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:03 smithi177 ceph-mon[33043]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:10:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:03 smithi098 ceph-mon[32569]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/1977850749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806b0f0 msgr2=0x7f7d0810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/1977850749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806b0f0 0x7f7d0810f900 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b0a0 tx=0x7f7d0000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/1977850749 shutdown_connections 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/1977850749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806b0f0 0x7f7d0810f900 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/1977850749 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d08108a30 0x7f7d08108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/1977850749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d0806a750 0x7f7d0806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/1977850749 >> 172.21.15.37:0/1977850749 conn(0x7f7d080788f0 msgr2=0x7f7d08078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/1977850749 shutdown_connections 2024-04-25T21:10:04.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.778+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/1977850749 wait complete. 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 Processor -- start 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 -- start start 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d0806b0f0 0x7f7d08199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d08108a30 0x7f7d0819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:04.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.779+0000 7f7d0fa8f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112cb0 con 0x7f7d0806a750 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0fa8f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112b30 con 0x7f7d08108a30 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0fa8f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08112e30 con 0x7f7d0806b0f0 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40606/0 (socket says 172.21.15.37:40606) 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 -- 172.21.15.37:0/4167454340 learned_addr learned my addr 172.21.15.37:0/4167454340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d02a700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d0806b0f0 0x7f7d08199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0e02c700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d08108a30 0x7f7d0819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 -- 172.21.15.37:0/4167454340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d0806b0f0 msgr2=0x7f7d08199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d0806b0f0 0x7f7d08199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 -- 172.21.15.37:0/4167454340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d08108a30 msgr2=0x7f7d0819db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d08108a30 0x7f7d0819db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 -- 172.21.15.37:0/4167454340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0819e250 con 0x7f7d0806a750 2024-04-25T21:10:04.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0d82b700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b530 tx=0x7f7d0400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04015020 con 0x7f7d0806a750 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d04011ce0 con 0x7f7d0806a750 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0819e540 con 0x7f7d0806a750 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.780+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04010dd0 con 0x7f7d0806a750 2024-04-25T21:10:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.781+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081a5dc0 con 0x7f7d0806a750 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.781+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0810b030 con 0x7f7d0806a750 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.783+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d04018020 con 0x7f7d0806a750 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.783+0000 7f7cfeffd700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 0x7f7cf40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.783+0000 7f7d0d02a700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 0x7f7cf40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.783+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f7d040cfd00 con 0x7f7d0806a750 2024-04-25T21:10:04.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.784+0000 7f7d0d02a700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 0x7f7cf40a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d08078fd0 tx=0x7f7cf8009190 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:04.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:04.788+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d0401f020 con 0x7f7d0806a750 2024-04-25T21:10:05.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.043+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d0810aa10 con 0x7f7d0806a750 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.044+0000 7f7cfeffd700 1 -- 172.21.15.37:0/4167454340 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f7d0409c4e0 con 0x7f7d0806a750 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:10:05.044 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.046+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 msgr2=0x7f7cf40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.046+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 0x7f7cf40a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7d08078fd0 tx=0x7f7cf8009190 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.046+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 msgr2=0x7f7d08199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400b530 tx=0x7f7d0400b840 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 shutdown_connections 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7cf40a3e30 0x7f7cf40a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d08108a30 0x7f7d0819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d0806a750 0x7f7d08199240 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 --2- 172.21.15.37:0/4167454340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7d0806b0f0 0x7f7d08199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 >> 172.21.15.37:0/4167454340 conn(0x7f7d080788f0 msgr2=0x7f7d080ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:05.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 shutdown_connections 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:05.047+0000 7f7d0fa8f700 1 -- 172.21.15.37:0/4167454340 wait complete. 2024-04-25T21:10:05.048 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:10:05.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:05 smithi037 ceph-mon[29195]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:10:05.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:05 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4167454340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:05 smithi177 ceph-mon[33043]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:10:05.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:05 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4167454340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:05 smithi098 ceph-mon[32569]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-25T21:10:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:05 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4167454340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:05.800 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 2bb2dd22 --release --await' 2024-04-25T21:10:05.800 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 2bb2dd22 --release --await 2024-04-25T21:10:06.054 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:07.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:07 smithi037 ceph-mon[29195]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:07.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:07 smithi177 ceph-mon[33043]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:07 smithi098 ceph-mon[32569]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:07.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 -- 172.21.15.37:0/867587176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8102a90 msgr2=0x7fc8d8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 --2- 172.21.15.37:0/867587176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8102a90 0x7fc8d8102f10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009990 tx=0x7fc8cc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 -- 172.21.15.37:0/867587176 shutdown_connections 2024-04-25T21:10:07.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 --2- 172.21.15.37:0/867587176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8103450 0x7fc8d810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 --2- 172.21.15.37:0/867587176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 0x7fc8d8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 --2- 172.21.15.37:0/867587176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8102a90 0x7fc8d8102f10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 -- 172.21.15.37:0/867587176 >> 172.21.15.37:0/867587176 conn(0x7fc8d8079ba0 msgr2=0x7fc8d8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 -- 172.21.15.37:0/867587176 shutdown_connections 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.876+0000 7fc8de289700 1 -- 172.21.15.37:0/867587176 wait complete. 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.877+0000 7fc8de289700 1 Processor -- start 2024-04-25T21:10:07.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.877+0000 7fc8de289700 1 -- start start 2024-04-25T21:10:07.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8102a90 0x7fc8d81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 0x7fc8d819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8112c80 con 0x7fc8d8103450 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8112b00 con 0x7fc8d8108a50 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8de289700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8112e00 con 0x7fc8d8102a90 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40624/0 (socket says 172.21.15.37:40624) 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8102a90 0x7fc8d81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 -- 172.21.15.37:0/942698525 learned_addr learned my addr 172.21.15.37:0/942698525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8dc826700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 0x7fc8d819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 -- 172.21.15.37:0/942698525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8102a90 msgr2=0x7fc8d81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8102a90 0x7fc8d81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 -- 172.21.15.37:0/942698525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 msgr2=0x7fc8d819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 0x7fc8d819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 -- 172.21.15.37:0/942698525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d819e1c0 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.878+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0057c0 tx=0x7fc8cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.879+0000 7fc8d57fa700 1 -- 172.21.15.37:0/942698525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc027070 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.879+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d819e450 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.879+0000 7fc8d57fa700 1 -- 172.21.15.37:0/942698525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc003f20 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.879+0000 7fc8d57fa700 1 -- 172.21.15.37:0/942698525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc022bc0 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.879+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d81a5d30 con 0x7fc8d8103450 2024-04-25T21:10:07.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.881+0000 7fc8d57fa700 1 -- 172.21.15.37:0/942698525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc8cc02a070 con 0x7fc8d8103450 2024-04-25T21:10:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.882+0000 7fc8d57fa700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 0x7fc8c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.882+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 0x7fc8c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.882+0000 7fc8d57fa700 1 -- 172.21.15.37:0/942698525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc8cc0d1750 con 0x7fc8d8103450 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.882+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 0x7fc8c00a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009a10 tx=0x7fc8c8009190 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.882+0000 7fc8e8b12b80 1 Processor -- start 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 -- start start 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfcba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55658df1ed90 con 0x55658dfcbf70 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55658dec5a30 con 0x55658dfd6e70 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55658df1ad50 con 0x55658dfcb650 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40640/0 (socket says 172.21.15.37:40640) 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 -- 172.21.15.37:0/967711692 learned_addr learned my addr 172.21.15.37:0/967711692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfcba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8dc826700 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 -- 172.21.15.37:0/967711692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 msgr2=0x55658dfcba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfcba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 -- 172.21.15.37:0/967711692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 msgr2=0x55658dfd9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 -- 172.21.15.37:0/967711692 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658de89840 con 0x55658dfcbf70 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.883+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0a5e00 tx=0x7fc8cc0226c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b52eaa0741368e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8be7fc700 1 -- 172.21.15.37:0/967711692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0a1b00 con 0x55658dfcbf70 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8be7fc700 1 -- 172.21.15.37:0/967711692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc0a6330 con 0x55658dfcbf70 2024-04-25T21:10:07.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8be7fc700 1 -- 172.21.15.37:0/967711692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0a5ea0 con 0x55658dfcbf70 2024-04-25T21:10:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/967711692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 msgr2=0x55658dfd6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0a5e00 tx=0x7fc8cc0226c0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/967711692 shutdown_connections 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfcba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/967711692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd6930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/967711692 >> 172.21.15.37:0/967711692 conn(0x55658def8990 msgr2=0x55658dfc7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/967711692 shutdown_connections 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.884+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/967711692 wait complete. 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 -- start start 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 Processor -- start 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfd6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55658df1ed90 con 0x55658dfcbf70 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55658ded86b0 con 0x55658dfd6e70 2024-04-25T21:10:07.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55658dfd97a0 con 0x55658dfcb650 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.885+0000 7fc8e8b12b80 10 client.0 _ll_register_callbacks cb 0x55658deb7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40650/0 (socket says 172.21.15.37:40650) 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 -- 172.21.15.37:0/643288726 learned_addr learned my addr 172.21.15.37:0/643288726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8dc826700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 -- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 msgr2=0x55658dfd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 -- 172.21.15.37:0/643288726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 msgr2=0x55658dfd3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 -- 172.21.15.37:0/643288726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658de6c9d0 con 0x55658dfcbf70 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0cf3f0 tx=0x7fc8cc0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f46a3ccb03da1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc09b7f0 con 0x55658dfcbf70 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8cc0a6330 con 0x55658dfcbf70 2024-04-25T21:10:07.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc8cc0a60b0 con 0x55658dfcbf70 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8cc0a1b00 con 0x55658dfcbf70 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8e8b12b80 10 client.15591 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.886+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55658dfe3fb0 con 0x55658dfcbf70 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc0a6330 con 0x55658dfcbf70 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8cc0b6070 con 0x55658dfcbf70 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 10 client.15591 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 20 client.15591 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658dfe6480 0x55658dfe8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 4 client.15591 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658df2c580 con 0x55658dfe6480 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.887+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658dfe6480 0x55658dfe8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.888+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658dfe6480 0x55658dfe8870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5bf418687d41fb5 server_cookie=d58ddb2052266470 in_seq=0 out_seq=0 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.888+0000 7fc8b77fe700 10 client.15591 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:07.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.888+0000 7fc8b77fe700 1 -- 172.21.15.37:0/643288726 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55658df2c580 con 0x55658dfe6480 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.888+0000 7fc8b77fe700 10 client.15591 handle_command_reply: tid=0 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 client.15591 shutdown 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 msgr2=0x55658dfd39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0cf3f0 tx=0x7fc8cc0092f0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 shutdown_connections 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658dfe6480 0x55658dfe8870 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfd3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8b77fe700 0 client.15591 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/643288726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfd39a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 >> 172.21.15.37:0/643288726 conn(0x55658def8990 msgr2=0x55658dfcdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 shutdown_connections 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/643288726 wait complete. 2024-04-25T21:10:07.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.889+0000 7fc8e8b12b80 20 client.15591 trim_cache size 0 max 16384 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.908+0000 7fc8e8b12b80 1 Processor -- start 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 -- start start 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfe2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfc7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81ad1d0 con 0x55658dfcbf70 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81aed10 con 0x55658dfd6e70 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81ad8a0 con 0x55658dfcb650 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40654/0 (socket says 172.21.15.37:40654) 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 -- 172.21.15.37:0/2782356784 learned_addr learned my addr 172.21.15.37:0/2782356784 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfe2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 -- 172.21.15.37:0/2782356784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 msgr2=0x55658dfe2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfe2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8dc826700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfc7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 -- 172.21.15.37:0/2782356784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 msgr2=0x55658dfc7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfc7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 -- 172.21.15.37:0/2782356784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658dfd95a0 con 0x55658dfcbf70 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfe2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.909+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0880b0 tx=0x7fc8cc09bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e7ac41e01f44ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8bdffb700 1 -- 172.21.15.37:0/2782356784 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0a6330 con 0x55658dfcbf70 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8bdffb700 1 -- 172.21.15.37:0/2782356784 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc09d750 con 0x55658dfcbf70 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8bdffb700 1 -- 172.21.15.37:0/2782356784 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc099590 con 0x55658dfcbf70 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/2782356784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 msgr2=0x55658dfc73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0880b0 tx=0x7fc8cc09bc60 comp rx=0 tx=0).stop 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/2782356784 shutdown_connections 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658dfc7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcb650 0x55658dfe2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/2782356784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcbf70 0x55658dfc73c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/2782356784 >> 172.21.15.37:0/2782356784 conn(0x55658def8990 msgr2=0x55658dfdde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/2782356784 shutdown_connections 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.910+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/2782356784 wait complete. 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 -- start start 2024-04-25T21:10:07.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 Processor -- start 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcbf70 0x55658e04a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658e044790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81ae640 con 0x55658dfcb650 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81adf70 con 0x55658dfd6e70 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81ab190 con 0x55658dfcbf70 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8e8b12b80 10 client.0 _ll_register_callbacks cb 0x55658deb7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40656/0 (socket says 172.21.15.37:40656) 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d7fff700 1 -- 172.21.15.37:0/4132183588 learned_addr learned my addr 172.21.15.37:0/4132183588 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8dc826700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658e044790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcbf70 0x55658e04a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d7fff700 1 -- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcbf70 msgr2=0x55658e04a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.911+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcbf70 0x55658e04a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8d7fff700 1 -- 172.21.15.37:0/4132183588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 msgr2=0x55658e044790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658e044790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8d7fff700 1 -- 172.21.15.37:0/4132183588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658e044e70 con 0x55658dfcb650 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8d7fff700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55658dfe8ed0 tx=0x7fc8c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cd6d43c44dc2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c801b040 con 0x55658dfcb650 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fc8c8029460 con 0x55658dfcb650 2024-04-25T21:10:07.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8cc09d750 con 0x55658dfcb650 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8c80296a0 con 0x55658dfcb650 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 10 client.15597 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55658e0513a0 con 0x55658dfcb650 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8cc09d750 con 0x55658dfcb650 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc8c803b450 con 0x55658dfcb650 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 10 client.15597 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 20 client.15597 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.912+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658e051a60 0x55658e053e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.913+0000 7fc8e8b12b80 4 client.15597 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2bb2dd22", "release": true, "await": true}] 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.913+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2bb2dd22", "release": true, "await": true}) -- 0x55658df26620 con 0x55658e051a60 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.913+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658e051a60 0x55658e053e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.913+0000 7fc8d77fe700 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658e051a60 0x55658e053e80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785ad62245d7512c server_cookie=6da74a534d65ef5c in_seq=0 out_seq=0 2024-04-25T21:10:07.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.913+0000 7fc8b77fe700 10 client.15597 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.917+0000 7fc8b77fe700 1 -- 172.21.15.37:0/4132183588 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55658df26620 con 0x55658e051a60 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.917+0000 7fc8b77fe700 10 client.15597 handle_command_reply: tid=0 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.917+0000 7fc8e8b12b80 1 client.15597 shutdown 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 msgr2=0x55658e049ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x55658dfe8ed0 tx=0x7fc8c8004750 comp rx=0 tx=0).stop 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 shutdown_connections 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55658e051a60 0x55658e053e80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55658dfd6e70 0x55658e044790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8b77fe700 0 client.15597 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55658dfcb650 0x55658e049ba0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 --2- 172.21.15.37:0/4132183588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55658dfcbf70 0x55658e04a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 >> 172.21.15.37:0/4132183588 conn(0x55658def8990 msgr2=0x55658dfde690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 shutdown_connections 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 1 -- 172.21.15.37:0/4132183588 wait complete. 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8e8b12b80 20 client.15597 trim_cache size 0 max 16384 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:10:07.920 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "2bb2dd22": { 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.2, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:10:07.921 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:07.922 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 msgr2=0x7fc8c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 0x7fc8c00a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009a10 tx=0x7fc8c8009190 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 msgr2=0x7fc8d81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.918+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0057c0 tx=0x7fc8cc0057f0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 shutdown_connections 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc8c00a3db0 0x7fc8c00a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d8108a50 0x7fc8d819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8d8102a90 0x7fc8d81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 --2- 172.21.15.37:0/942698525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8d8103450 0x7fc8d81996f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 >> 172.21.15.37:0/942698525 conn(0x7fc8d8079ba0 msgr2=0x7fc8d81017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 shutdown_connections 2024-04-25T21:10:07.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:07.919+0000 7fc8de289700 1 -- 172.21.15.37:0/942698525 wait complete. 2024-04-25T21:10:07.924 INFO:tasks.workunit.client.0.smithi037.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-25T21:10:07.924 INFO:tasks.workunit.client.0.smithi037.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-25T21:10:07.924 INFO:tasks.workunit.client.0.smithi037.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-25T21:10:08.845 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2bb2dd22', seconds elapsed: 43.7 2024-04-25T21:10:08.845 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T21:10:08.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:10:09.105 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:09.387 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:09 smithi037 ceph-mon[29195]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:09 smithi177 ceph-mon[33043]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:09 smithi098 ceph-mon[32569]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T21:10:10.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 -- 172.21.15.37:0/3471706170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488108ab0 msgr2=0x7f2488108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 --2- 172.21.15.37:0/3471706170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488108ab0 0x7f2488108e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2478009990 tx=0x7f2478009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 -- 172.21.15.37:0/3471706170 shutdown_connections 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 --2- 172.21.15.37:0/3471706170 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488103480 0x7f248810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 --2- 172.21.15.37:0/3471706170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488108ab0 0x7f2488108e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 --2- 172.21.15.37:0/3471706170 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488102ac0 0x7f2488102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 -- 172.21.15.37:0/3471706170 >> 172.21.15.37:0/3471706170 conn(0x7f2488079bf0 msgr2=0x7f248807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 -- 172.21.15.37:0/3471706170 shutdown_connections 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.946+0000 7f248da75700 1 -- 172.21.15.37:0/3471706170 wait complete. 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 Processor -- start 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 -- start start 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488103480 0x7f2488075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488108ab0 0x7f24880757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2488112cb0 con 0x7f2488102ac0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2488112b30 con 0x7f2488103480 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f248da75700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2488112e30 con 0x7f2488108ab0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f24877fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f24877fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56042/0 (socket says 172.21.15.37:56042) 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f24877fe700 1 -- 172.21.15.37:0/2275580291 learned_addr learned my addr 172.21.15.37:0/2275580291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f2486ffd700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488103480 0x7f2488075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.947+0000 7f2487fff700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488108ab0 0x7f24880757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 -- 172.21.15.37:0/2275580291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488108ab0 msgr2=0x7f24880757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488108ab0 0x7f24880757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 -- 172.21.15.37:0/2275580291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488103480 msgr2=0x7f2488075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488103480 0x7f2488075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 -- 172.21.15.37:0/2275580291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24881a7730 con 0x7f2488102ac0 2024-04-25T21:10:10.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f24877fe700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2478000c00 tx=0x7f2478009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:10.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478027070 con 0x7f2488102ac0 2024-04-25T21:10:10.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2478022470 con 0x7f2488102ac0 2024-04-25T21:10:10.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24881a79c0 con 0x7f2488102ac0 2024-04-25T21:10:10.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2478017650 con 0x7f2488102ac0 2024-04-25T21:10:10.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.948+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24881a7e70 con 0x7f2488102ac0 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.949+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24881041f0 con 0x7f2488102ac0 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.950+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f247802a070 con 0x7f2488102ac0 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.951+0000 7f2484ff9700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 0x7f24700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.951+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f24780d1420 con 0x7f2488102ac0 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.951+0000 7f2486ffd700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 0x7f24700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:10.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.951+0000 7f2486ffd700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 0x7f24700a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f248807a310 tx=0x7f247c00b410 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:10.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:10.956+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2478013070 con 0x7f2488102ac0 2024-04-25T21:10:11.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.210+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248802df80 con 0x7f2488102ac0 2024-04-25T21:10:11.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.210+0000 7f2484ff9700 1 -- 172.21.15.37:0/2275580291 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f247809db80 con 0x7f2488102ac0 2024-04-25T21:10:11.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:10:11.211 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:10:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 msgr2=0x7f24700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 0x7f24700a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f248807a310 tx=0x7f247c00b410 comp rx=0 tx=0).stop 2024-04-25T21:10:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 msgr2=0x7f2488074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2478000c00 tx=0x7f2478009580 comp rx=0 tx=0).stop 2024-04-25T21:10:11.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 shutdown_connections 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f24700a3e30 0x7f24700a62f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2488108ab0 0x7f24880757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2488102ac0 0x7f2488074d40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 --2- 172.21.15.37:0/2275580291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2488103480 0x7f2488075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 >> 172.21.15.37:0/2275580291 conn(0x7f2488079bf0 msgr2=0x7f2488100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 shutdown_connections 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:11.213+0000 7f248da75700 1 -- 172.21.15.37:0/2275580291 wait complete. 2024-04-25T21:10:11.214 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:10:11.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:11 smithi037 ceph-mon[29195]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-25T21:10:11.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:11 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2275580291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:11 smithi177 ceph-mon[33043]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-25T21:10:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:11 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2275580291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:11 smithi098 ceph-mon[32569]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-25T21:10:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:11 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2275580291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:12.063 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T21:10:12.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T21:10:12.318 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:13.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:13 smithi037 ceph-mon[29195]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-25T21:10:13.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:13 smithi177 ceph-mon[33043]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-25T21:10:13.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:13 smithi098 ceph-mon[32569]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 -- 172.21.15.37:0/1452937043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f81034b0 msgr2=0x7f17f810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 --2- 172.21.15.37:0/1452937043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f81034b0 0x7f17f810f8e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f17f000b0d0 tx=0x7f17f000b3e0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 -- 172.21.15.37:0/1452937043 shutdown_connections 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 --2- 172.21.15.37:0/1452937043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f81034b0 0x7f17f810f8e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 --2- 172.21.15.37:0/1452937043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8108a30 0x7f17f8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 --2- 172.21.15.37:0/1452937043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f8102af0 0x7f17f8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 -- 172.21.15.37:0/1452937043 >> 172.21.15.37:0/1452937043 conn(0x7f17f8079b10 msgr2=0x7f17f8079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 -- 172.21.15.37:0/1452937043 shutdown_connections 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 -- 172.21.15.37:0/1452937043 wait complete. 2024-04-25T21:10:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.165+0000 7f17fd986700 1 Processor -- start 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 -- start start 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8102af0 0x7f17f819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 0x7f17f819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8112c00 con 0x7f17f8108a30 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8112a80 con 0x7f17f81034b0 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17fd986700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17f8112d80 con 0x7f17f8102af0 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56060/0 (socket says 172.21.15.37:56060) 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 -- 172.21.15.37:0/463591458 learned_addr learned my addr 172.21.15.37:0/463591458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17effff700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 0x7f17f819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8102af0 0x7f17f819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 -- 172.21.15.37:0/463591458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8102af0 msgr2=0x7f17f819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8102af0 0x7f17f819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 -- 172.21.15.37:0/463591458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 msgr2=0x7f17f819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 0x7f17f819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17f77fe700 1 -- 172.21.15.37:0/463591458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f81a0270 con 0x7f17f8108a30 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.166+0000 7f17effff700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 0x7f17f819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17f77fe700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17f00095c0 tx=0x7f17f0007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17fc984700 1 -- 172.21.15.37:0/463591458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001f070 con 0x7f17f8108a30 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f81a0500 con 0x7f17f8108a30 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81a7de0 con 0x7f17f8108a30 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17fc984700 1 -- 172.21.15.37:0/463591458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f0003c00 con 0x7f17f8108a30 2024-04-25T21:10:14.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.167+0000 7f17fc984700 1 -- 172.21.15.37:0/463591458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001b6a0 con 0x7f17f8108a30 2024-04-25T21:10:14.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.169+0000 7f17fc984700 1 -- 172.21.15.37:0/463591458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17f0022070 con 0x7f17f8108a30 2024-04-25T21:10:14.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.170+0000 7f17fc984700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 0x7f17e40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.170+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 0x7f17e40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.170+0000 7f17fc984700 1 -- 172.21.15.37:0/463591458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f17f000f020 con 0x7f17f8108a30 2024-04-25T21:10:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 0x7f17e40a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17e800dad0 tx=0x7f17e800d4b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 Processor -- start 2024-04-25T21:10:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 -- start start 2024-04-25T21:10:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539d2a80 0x5596539d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539de270 0x5596539e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559653925ec0 con 0x5596539d33a0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5596538ccaf0 con 0x5596539d2a80 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f180820fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559653921e80 con 0x5596539de270 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f17effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f17effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56062/0 (socket says 172.21.15.37:56062) 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.171+0000 7f17effff700 1 -- 172.21.15.37:0/2792019150 learned_addr learned my addr 172.21.15.37:0/2792019150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17f77fe700 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539de270 0x5596539e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539d2a80 0x5596539d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 -- 172.21.15.37:0/2792019150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539de270 msgr2=0x5596539e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539de270 0x5596539e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 -- 172.21.15.37:0/2792019150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539d2a80 msgr2=0x5596539d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539d2a80 0x5596539d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 -- 172.21.15.37:0/2792019150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559653873a60 con 0x5596539d33a0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17effff700 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17e0008c50 tx=0x7f17e0008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6411ecc01bf0db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17edffb700 1 -- 172.21.15.37:0/2792019150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0014070 con 0x5596539d33a0 2024-04-25T21:10:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17edffb700 1 -- 172.21.15.37:0/2792019150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e000ed40 con 0x5596539d33a0 2024-04-25T21:10:14.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.172+0000 7f17edffb700 1 -- 172.21.15.37:0/2792019150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0005680 con 0x5596539d33a0 2024-04-25T21:10:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- 172.21.15.37:0/2792019150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 msgr2=0x5596539ddd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f17e0008c50 tx=0x7f17e0008f60 comp rx=0 tx=0).stop 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- 172.21.15.37:0/2792019150 shutdown_connections 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539de270 0x5596539e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539d2a80 0x5596539d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 --2- 172.21.15.37:0/2792019150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539ddd30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- 172.21.15.37:0/2792019150 >> 172.21.15.37:0/2792019150 conn(0x559653900820 msgr2=0x5596539cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- 172.21.15.37:0/2792019150 shutdown_connections 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- 172.21.15.37:0/2792019150 wait complete. 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.173+0000 7f180820fb80 1 -- start start 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 Processor -- start 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539db1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596539e0c60 con 0x5596539d2a80 2024-04-25T21:10:14.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5596538c4d60 con 0x5596539de270 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5596539e0de0 con 0x5596539d33a0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f180820fb80 10 client.0 _ll_register_callbacks cb 0x5596538beae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56070/0 (socket says 172.21.15.37:56070) 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 -- 172.21.15.37:0/4078591485 learned_addr learned my addr 172.21.15.37:0/4078591485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17effff700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f77fe700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539db1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 -- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 msgr2=0x5596539dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 -- 172.21.15.37:0/4078591485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 msgr2=0x5596539db1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539db1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 -- 172.21.15.37:0/4078591485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596538dd400 con 0x5596539d2a80 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ff60 tx=0x7f17e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0548da6dcf8f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.174+0000 7f17d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:14.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8028670 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f17e8028810 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e000ed40 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17e8028a10 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 10 client.15615 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5596539eb4e0 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e000ed40 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17e801f430 con 0x5596539d2a80 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 10 client.15615 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 20 client.15615 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5596539eda70 0x5596539efe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 4 client.15615 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.175+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596539336b0 con 0x5596539eda70 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.176+0000 7f17effff700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5596539eda70 0x5596539efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.176+0000 7f17effff700 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5596539eda70 0x5596539efe80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884d48cff29bd37b server_cookie=51f8b0667428cbcf in_seq=0 out_seq=0 2024-04-25T21:10:14.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.176+0000 7f17d6ffd700 10 client.15615 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f17d6ffd700 1 -- 172.21.15.37:0/4078591485 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596539336b0 con 0x5596539eda70 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f17d6ffd700 10 client.15615 handle_command_reply: tid=0 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 client.15615 shutdown 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 msgr2=0x5596539dd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f17e800ff60 tx=0x7f17e8004750 comp rx=0 tx=0).stop 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 shutdown_connections 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5596539eda70 0x5596539efe80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539db1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539dd990 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f17d6ffd700 0 client.15615 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 --2- 172.21.15.37:0/4078591485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 >> 172.21.15.37:0/4078591485 conn(0x559653900820 msgr2=0x5596539ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 shutdown_connections 2024-04-25T21:10:14.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 1 -- 172.21.15.37:0/4078591485 wait complete. 2024-04-25T21:10:14.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.177+0000 7f180820fb80 20 client.15615 trim_cache size 0 max 16384 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 Processor -- start 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 -- start start 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 0x5596539f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81af320 con 0x5596539d33a0 2024-04-25T21:10:14.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b0e60 con 0x5596539de270 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f180820fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81af9f0 con 0x5596539d2a80 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56076/0 (socket says 172.21.15.37:56076) 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 -- 172.21.15.37:0/1843674981 learned_addr learned my addr 172.21.15.37:0/1843674981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17f77fe700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 0x5596539f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 -- 172.21.15.37:0/1843674981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 msgr2=0x5596539f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 0x5596539f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 -- 172.21.15.37:0/1843674981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 msgr2=0x5596539e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17effff700 1 -- 172.21.15.37:0/1843674981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596539d4450 con 0x5596539d33a0 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 0x5596539f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.197+0000 7f17f77fe700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:14.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f17effff700 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17e0008fd0 tx=0x7f17e001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7032fc2087eec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f17ed7fa700 1 -- 172.21.15.37:0/1843674981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0014040 con 0x5596539d33a0 2024-04-25T21:10:14.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f17ed7fa700 1 -- 172.21.15.37:0/1843674981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e000ed00 con 0x5596539d33a0 2024-04-25T21:10:14.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f17ed7fa700 1 -- 172.21.15.37:0/1843674981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0004280 con 0x5596539d33a0 2024-04-25T21:10:14.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f180820fb80 1 -- 172.21.15.37:0/1843674981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 msgr2=0x5596539f5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.198+0000 7f180820fb80 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f17e0008fd0 tx=0x7f17e001b040 comp rx=0 tx=0).stop 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 -- 172.21.15.37:0/1843674981 shutdown_connections 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d2a80 0x5596539f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 --2- 172.21.15.37:0/1843674981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d33a0 0x5596539f5c90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 -- 172.21.15.37:0/1843674981 >> 172.21.15.37:0/1843674981 conn(0x559653900820 msgr2=0x5596539dcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 -- 172.21.15.37:0/1843674981 shutdown_connections 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 -- 172.21.15.37:0/1843674981 wait complete. 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 -- start start 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 Processor -- start 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.199+0000 7f180820fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b0790 con 0x5596539d2a80 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81b00c0 con 0x5596539de270 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f17f81ad2c0 con 0x5596539d33a0 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 10 client.0 _ll_register_callbacks cb 0x5596538beae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56092/0 (socket says 172.21.15.37:56092) 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 -- 172.21.15.37:0/565647522 learned_addr learned my addr 172.21.15.37:0/565647522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f77fe700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17effff700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 -- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 msgr2=0x5596539e3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539e3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 -- 172.21.15.37:0/565647522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 msgr2=0x5596539e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 -- 172.21.15.37:0/565647522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559653a53fd0 con 0x5596539d2a80 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17effff700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539e3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:14.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f77fe700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17f6ffd700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17e801f800 tx=0x7f17e8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a409d31b71e0cde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e801fc80 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f17e8028db0 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e000ed00 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.200+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f17e8028470 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 10 client.15621 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559653a55140 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e000ed00 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f17e8039030 con 0x5596539d2a80 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 10 client.15621 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 20 client.15621 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559653a558f0 0x559653a57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 4 client.15621 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T21:10:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x55965392d750 con 0x559653a558f0 2024-04-25T21:10:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.201+0000 7f17effff700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559653a558f0 0x559653a57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.202+0000 7f17effff700 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559653a558f0 0x559653a57d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c313773559a3ece8 server_cookie=5af7631de33c6494 in_seq=0 out_seq=0 2024-04-25T21:10:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.202+0000 7f17d6ffd700 10 client.15621 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:14.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.216+0000 7f17d6ffd700 1 -- 172.21.15.37:0/565647522 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55965392d750 con 0x559653a558f0 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f17d6ffd700 10 client.15621 handle_command_reply: tid=0 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 client.15621 shutdown 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 msgr2=0x5596539e3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f17e801f800 tx=0x7f17e8011ab0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 shutdown_connections 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559653a558f0 0x559653a57d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5596539de270 0x5596539e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5596539d2a80 0x5596539e3000 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 --2- 172.21.15.37:0/565647522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596539d33a0 0x5596539e3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 >> 172.21.15.37:0/565647522 conn(0x559653900820 msgr2=0x5596539dcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f17d6ffd700 0 client.15621 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 shutdown_connections 2024-04-25T21:10:14.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 1 -- 172.21.15.37:0/565647522 wait complete. 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.217+0000 7f180820fb80 20 client.15621 trim_cache size 0 max 16384 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "bb93a79b": { 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:14.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.2, 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:14.220 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 msgr2=0x7f17e40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 0x7f17e40a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17e800dad0 tx=0x7f17e800d4b0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 msgr2=0x7f17f819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f17f00095c0 tx=0x7f17f0007ad0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 shutdown_connections 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f17e40a3ed0 0x7f17e40a6390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17f8108a30 0x7f17f819fb30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17f8102af0 0x7f17f819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 --2- 172.21.15.37:0/463591458 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17f81034b0 0x7f17f819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 >> 172.21.15.37:0/463591458 conn(0x7f17f8079b10 msgr2=0x7f17f810dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 shutdown_connections 2024-04-25T21:10:14.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:14.218+0000 7f17fd986700 1 -- 172.21.15.37:0/463591458 wait complete. 2024-04-25T21:10:14.983 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bb93a79b', quiesce took 6.1 seconds. Will release after: 26.2 2024-04-25T21:10:15.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:15 smithi037 ceph-mon[29195]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-25T21:10:15.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:15 smithi177 ceph-mon[33043]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-25T21:10:15.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:15 smithi098 ceph-mon[32569]: pgmap v56: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-25T21:10:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 -- 172.21.15.37:0/241863820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc107270 msgr2=0x7f4fbc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 --2- 172.21.15.37:0/241863820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc107270 0x7f4fbc10ee40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009ca0 tx=0x7f4fb402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 -- 172.21.15.37:0/241863820 shutdown_connections 2024-04-25T21:10:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 --2- 172.21.15.37:0/241863820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc107270 0x7f4fbc10ee40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 --2- 172.21.15.37:0/241863820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc102b50 0x7f4fbc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 --2- 172.21.15.37:0/241863820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 -- 172.21.15.37:0/241863820 >> 172.21.15.37:0/241863820 conn(0x7f4fbc0793e0 msgr2=0x7f4fbc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 -- 172.21.15.37:0/241863820 shutdown_connections 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.003+0000 7f4fc4519640 1 -- 172.21.15.37:0/241863820 wait complete. 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 Processor -- start 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 -- start start 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc107270 0x7f4fbc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1121c0 con 0x7f4fbc074cd0 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112040 con 0x7f4fbc1021a0 2024-04-25T21:10:16.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc4519640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc112340 con 0x7f4fbc107270 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56112/0 (socket says 172.21.15.37:56112) 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 -- 172.21.15.37:0/2420666927 learned_addr learned my addr 172.21.15.37:0/2420666927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc107270 0x7f4fbc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 -- 172.21.15.37:0/2420666927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc107270 msgr2=0x7f4fbc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc107270 0x7f4fbc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 -- 172.21.15.37:0/2420666927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 msgr2=0x7f4fbc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc2ab6640 1 -- 172.21.15.37:0/2420666927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc1a6e90 con 0x7f4fbc074cd0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.004+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fc2ab6640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098d0 tx=0x7f4fb40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fb37fe640 1 -- 172.21.15.37:0/2420666927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4003b70 con 0x7f4fbc074cd0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fb37fe640 1 -- 172.21.15.37:0/2420666927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4003d10 con 0x7f4fbc074cd0 2024-04-25T21:10:16.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc1a7120 con 0x7f4fbc074cd0 2024-04-25T21:10:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fb37fe640 1 -- 172.21.15.37:0/2420666927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb402f9d0 con 0x7f4fbc074cd0 2024-04-25T21:10:16.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.005+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1a7670 con 0x7f4fbc074cd0 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.007+0000 7f4fb37fe640 1 -- 172.21.15.37:0/2420666927 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fb402fb90 con 0x7f4fbc074cd0 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 Processor -- start 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 -- start start 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb21a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb2ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e15ea86970 con 0x55e15eb21f40 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e15ea234a0 con 0x55e15eb21620 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.008+0000 7f4fc722e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e15ea82930 con 0x55e15eb2c800 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fb37fe640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 0x7f4fa80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc22b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc22b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56116/0 (socket says 172.21.15.37:56116) 2024-04-25T21:10:16.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc22b5640 1 -- 172.21.15.37:0/843197536 learned_addr learned my addr 172.21.15.37:0/843197536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb2ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb21a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc2ab6640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 0x7f4fa80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fb37fe640 1 -- 172.21.15.37:0/2420666927 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4fb40f1bc0 con 0x7f4fbc074cd0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/843197536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 msgr2=0x55e15eb21a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb21a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/843197536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 msgr2=0x55e15eb2c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/843197536 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e15e799210 con 0x55e15eb2c800 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb2ebf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc075c90 tx=0x7f4fac00a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b054b6c35205a113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fc2ab6640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 0x7f4fa80a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005cc0 tx=0x7f4fb4041b80 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.009+0000 7f4fb0bfe640 1 -- 172.21.15.37:0/843197536 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac005020 con 0x55e15eb2c800 2024-04-25T21:10:16.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fb0bfe640 1 -- 172.21.15.37:0/843197536 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0047a0 con 0x55e15eb2c800 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fb0bfe640 1 -- 172.21.15.37:0/843197536 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00f8a0 con 0x55e15eb2c800 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 -- 172.21.15.37:0/843197536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 msgr2=0x55e15eb2ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb2ebf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc075c90 tx=0x7f4fac00a680 comp rx=0 tx=0).stop 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 -- 172.21.15.37:0/843197536 shutdown_connections 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb2ebf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21f40 0x55e15eb2c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 --2- 172.21.15.37:0/843197536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb21a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 -- 172.21.15.37:0/843197536 >> 172.21.15.37:0/843197536 conn(0x55e15eb1db30 msgr2=0x55e15eb1df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 -- 172.21.15.37:0/843197536 shutdown_connections 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.010+0000 7f4fc722e740 1 -- 172.21.15.37:0/843197536 wait complete. 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 -- start start 2024-04-25T21:10:16.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 Processor -- start 2024-04-25T21:10:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb2c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb29420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e15eb2f430 con 0x55e15eb2c800 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e15ea210b0 con 0x55e15eb21620 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e15eb2f5b0 con 0x55e15eb21f40 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc722e740 10 client.0 _ll_register_callbacks cb 0x55e15ea0ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56122/0 (socket says 172.21.15.37:56122) 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/1281021598 learned_addr learned my addr 172.21.15.37:0/1281021598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb29420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.011+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb2c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 msgr2=0x55e15eb29420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb29420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/1281021598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 msgr2=0x55e15eb2c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb2c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/1281021598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e15e865620 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0046c0 tx=0x7f4fac01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71da82cfba56336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac022020 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4fac0054e0 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac0047a0 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fac011020 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc722e740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e15eb3b340 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0047a0 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.012+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fac010430 con 0x55e15eb2c800 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 20 client.15633 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eb3c6e0 0x55e15eb3ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 4 client.15633 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e15ea94390 con 0x55e15eb3c6e0 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eb3c6e0 0x55e15eb3ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fa5ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.013+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eb3c6e0 0x55e15eb3ead0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535dc53a15090ce0 server_cookie=72e898fb3d86a950 in_seq=0 out_seq=0 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.014+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/1281021598 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e15ea94390 con 0x55e15eb3c6e0 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.014+0000 7f4fa5ffb640 10 client.15633 handle_command_reply: tid=0 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.014+0000 7f4fc722e740 1 client.15633 shutdown 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 msgr2=0x55e15eb29960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0046c0 tx=0x7f4fac01a040 comp rx=0 tx=0).stop 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 shutdown_connections 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eb3c6e0 0x55e15eb3ead0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb2c800 0x55e15eb29960 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb29420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 --2- 172.21.15.37:0/1281021598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21620 0x55e15eb2c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fa5ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 >> 172.21.15.37:0/1281021598 conn(0x55e15eb1db30 msgr2=0x55e15eb238b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 shutdown_connections 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 1 -- 172.21.15.37:0/1281021598 wait complete. 2024-04-25T21:10:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.015+0000 7f4fc722e740 20 client.15633 trim_cache size 0 max 16384 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 Processor -- start 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 -- start start 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21f40 0x55e15eb1cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb44240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1afc50 con 0x55e15eb21620 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1b1790 con 0x55e15eb21f40 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc722e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1b0320 con 0x55e15eb2c800 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56124/0 (socket says 172.21.15.37:56124) 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3735618916 learned_addr learned my addr 172.21.15.37:0/3735618916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3735618916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 msgr2=0x55e15eb44240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb44240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3735618916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21f40 msgr2=0x55e15eb1cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21f40 0x55e15eb1cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21f40 0x55e15eb1cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.030+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3735618916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e15eb21d70 con 0x55e15eb21620 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4fac011fd0 tx=0x7f4fac004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3045a3859a9e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fa7fff640 1 -- 172.21.15.37:0/3735618916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac022070 con 0x55e15eb21620 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fa7fff640 1 -- 172.21.15.37:0/3735618916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac00a950 con 0x55e15eb21620 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fa7fff640 1 -- 172.21.15.37:0/3735618916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac010430 con 0x55e15eb21620 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 -- 172.21.15.37:0/3735618916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 msgr2=0x55e15eb382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4fac011fd0 tx=0x7f4fac004f30 comp rx=0 tx=0).stop 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 -- 172.21.15.37:0/3735618916 shutdown_connections 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb2c800 0x55e15eb44240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb21f40 0x55e15eb1cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3735618916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb382f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 -- 172.21.15.37:0/3735618916 >> 172.21.15.37:0/3735618916 conn(0x55e15eb1db30 msgr2=0x55e15eb1c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.031+0000 7f4fc722e740 1 -- 172.21.15.37:0/3735618916 shutdown_connections 2024-04-25T21:10:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 -- 172.21.15.37:0/3735618916 wait complete. 2024-04-25T21:10:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 -- start start 2024-04-25T21:10:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 Processor -- start 2024-04-25T21:10:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb2c800 0x55e15eb31a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1b10c0 con 0x55e15eb21620 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1b09f0 con 0x55e15eb2c800 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc1adc30 con 0x55e15eb21f40 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc722e740 10 client.0 _ll_register_callbacks cb 0x7f4fac0115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc1ab4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56128/0 (socket says 172.21.15.37:56128) 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3387007326 learned_addr learned my addr 172.21.15.37:0/3387007326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.032+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 msgr2=0x55e15eb9a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb9a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3387007326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb2c800 msgr2=0x55e15eb31a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb2c800 0x55e15eb31a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 -- 172.21.15.37:0/3387007326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e15eb32250 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc1ab4640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c64b26e3c81d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac022040 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4fac010a90 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac00a950 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4fac010cd0 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e15eba25e0 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac00a950 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4fac03e030 con 0x55e15eb21620 2024-04-25T21:10:16.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.033+0000 7f4fc722e740 20 client.15639 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eba2bf0 0x55e15eba4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fc722e740 4 client.15639 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e15eb29cf0 con 0x55e15eba2bf0 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eba2bf0 0x55e15eba4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fc22b5640 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eba2bf0 0x55e15eba4fe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923cfa4cf9b822ce server_cookie=e259adbc51d726a8 in_seq=0 out_seq=0 2024-04-25T21:10:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.034+0000 7f4fa5ffb640 10 client.15639 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.044+0000 7f4fa5ffb640 1 -- 172.21.15.37:0/3387007326 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e15eb29cf0 con 0x55e15eba2bf0 2024-04-25T21:10:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.044+0000 7f4fa5ffb640 10 client.15639 handle_command_reply: tid=0 2024-04-25T21:10:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.044+0000 7f4fc722e740 1 client.15639 shutdown 2024-04-25T21:10:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 msgr2=0x55e15eb99e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4fac000c00 tx=0x7f4fac004660 comp rx=0 tx=0).stop 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 shutdown_connections 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e15eba2bf0 0x55e15eba4fe0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e15eb2c800 0x55e15eb31a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fa5ffb640 0 client.15639 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e15eb21f40 0x55e15eb9a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 --2- 172.21.15.37:0/3387007326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e15eb21620 0x55e15eb99e20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 >> 172.21.15.37:0/3387007326 conn(0x55e15eb1db30 msgr2=0x55e15eb278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 shutdown_connections 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 1 -- 172.21.15.37:0/3387007326 wait complete. 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc722e740 20 client.15639 trim_cache size 0 max 16384 2024-04-25T21:10:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:10:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:10:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:10:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 msgr2=0x7f4fa80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 0x7f4fa80a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005cc0 tx=0x7f4fb4041b80 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 msgr2=0x7f4fbc1a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098d0 tx=0x7f4fb40317b0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 shutdown_connections 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4fbc074cd0 0x7f4fbc1a67f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4fa80a3dc0 0x7f4fa80a6280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fbc107270 0x7f4fbc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 --2- 172.21.15.37:0/2420666927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4fbc1021a0 0x7f4fbc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.045+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 >> 172.21.15.37:0/2420666927 conn(0x7f4fbc0793e0 msgr2=0x7f4fbc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.046+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 shutdown_connections 2024-04-25T21:10:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:16.046+0000 7f4fc4519640 1 -- 172.21.15.37:0/2420666927 wait complete. 2024-04-25T21:10:17.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:17 smithi037 ceph-mon[29195]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:17.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:17 smithi177 ceph-mon[33043]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:17 smithi098 ceph-mon[32569]: pgmap v57: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:19.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:19 smithi037 ceph-mon[29195]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:19 smithi177 ceph-mon[33043]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:19.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:19 smithi098 ceph-mon[32569]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-25T21:10:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:21 smithi177 ceph-mon[33043]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T21:10:21.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:21 smithi098 ceph-mon[32569]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T21:10:21.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:21 smithi037 ceph-mon[29195]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-25T21:10:23.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:23 smithi177 ceph-mon[33043]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T21:10:23.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:23 smithi098 ceph-mon[32569]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T21:10:23.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:23 smithi037 ceph-mon[29195]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-04-25T21:10:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:25 smithi177 ceph-mon[33043]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T21:10:25.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:25 smithi098 ceph-mon[32569]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T21:10:25.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:25 smithi037 ceph-mon[29195]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-25T21:10:27.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:27 smithi177 ceph-mon[33043]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:27 smithi098 ceph-mon[32569]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:27.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:27 smithi037 ceph-mon[29195]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 -- 172.21.15.37:0/1071023248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c102b70 msgr2=0x7f4a3c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 --2- 172.21.15.37:0/1071023248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c102b70 0x7f4a3c106bb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009900 tx=0x7f4a3002a410 comp rx=0 tx=0).stop 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 -- 172.21.15.37:0/1071023248 shutdown_connections 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 --2- 172.21.15.37:0/1071023248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c107290 0x7f4a3c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 --2- 172.21.15.37:0/1071023248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c102b70 0x7f4a3c106bb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 --2- 172.21.15.37:0/1071023248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c1021c0 0x7f4a3c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 -- 172.21.15.37:0/1071023248 >> 172.21.15.37:0/1071023248 conn(0x7f4a3c0fda20 msgr2=0x7f4a3c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 -- 172.21.15.37:0/1071023248 shutdown_connections 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.149+0000 7f4a416fa640 1 -- 172.21.15.37:0/1071023248 wait complete. 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 Processor -- start 2024-04-25T21:10:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 -- start start 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c1021c0 0x7f4a3c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c107290 0x7f4a3c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1121b0 con 0x7f4a3c074cd0 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c112030 con 0x7f4a3c1021c0 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a416fa640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c112330 con 0x7f4a3c107290 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53304/0 (socket says 172.21.15.37:53304) 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a3b7fe640 1 -- 172.21.15.37:0/3447536675 learned_addr learned my addr 172.21.15.37:0/3447536675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c107290 0x7f4a3c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.150+0000 7f4a3affd640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c1021c0 0x7f4a3c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 -- 172.21.15.37:0/3447536675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c107290 msgr2=0x7f4a3c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c107290 0x7f4a3c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 -- 172.21.15.37:0/3447536675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c1021c0 msgr2=0x7f4a3c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c1021c0 0x7f4a3c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 -- 172.21.15.37:0/3447536675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c1a6f80 con 0x7f4a3c074cd0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a3b7fe640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400eaa0 tx=0x7f4a3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a23fff640 1 -- 172.21.15.37:0/3447536675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400cdb0 con 0x7f4a3c074cd0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a23fff640 1 -- 172.21.15.37:0/3447536675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a34005020 con 0x7f4a3c074cd0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c1a7270 con 0x7f4a3c074cd0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a23fff640 1 -- 172.21.15.37:0/3447536675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a340116d0 con 0x7f4a3c074cd0 2024-04-25T21:10:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.151+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a7740 con 0x7f4a3c074cd0 2024-04-25T21:10:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.153+0000 7f4a23fff640 1 -- 172.21.15.37:0/3447536675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a340051d0 con 0x7f4a3c074cd0 2024-04-25T21:10:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 Processor -- start 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 -- start start 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f2fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2ff40 0x557eb1f3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557eb1e94970 con 0x557eb1f3a800 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557eb1e314a0 con 0x557eb1f2f620 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a44415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557eb1e90930 con 0x557eb1f2ff40 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a23fff640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 0x7f4a240a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53308/0 (socket says 172.21.15.37:53308) 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3a7fc640 1 -- 172.21.15.37:0/3413306703 learned_addr learned my addr 172.21.15.37:0/3413306703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3affd640 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2ff40 0x557eb1f3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3b7fe640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 0x7f4a240a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.154+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f2fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a23fff640 1 -- 172.21.15.37:0/3447536675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4a340d08d0 con 0x7f4a3c074cd0 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 -- 172.21.15.37:0/3413306703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2ff40 msgr2=0x557eb1f3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2ff40 0x557eb1f3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 -- 172.21.15.37:0/3413306703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 msgr2=0x557eb1f2fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f2fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 -- 172.21.15.37:0/3413306703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb1ba7210 con 0x557eb1f3a800 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3b7fe640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 0x7f4a240a62b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a34066f70 tx=0x7f4a34067440 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009900 tx=0x7f4a30005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbc0abfa5dfb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a217fa640 1 -- 172.21.15.37:0/3413306703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30035070 con 0x557eb1f3a800 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a217fa640 1 -- 172.21.15.37:0/3413306703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a300035f0 con 0x557eb1f3a800 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.155+0000 7f4a217fa640 1 -- 172.21.15.37:0/3413306703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a300044e0 con 0x557eb1f3a800 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- 172.21.15.37:0/3413306703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 msgr2=0x557eb1f3cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009900 tx=0x7f4a30005780 comp rx=0 tx=0).stop 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- 172.21.15.37:0/3413306703 shutdown_connections 2024-04-25T21:10:29.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f3a800 0x557eb1f3cbf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2ff40 0x557eb1f3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 --2- 172.21.15.37:0/3413306703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f2fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- 172.21.15.37:0/3413306703 >> 172.21.15.37:0/3413306703 conn(0x557eb1f2bb30 msgr2=0x557eb1f2bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- 172.21.15.37:0/3413306703 shutdown_connections 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- 172.21.15.37:0/3413306703 wait complete. 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 -- start start 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.156+0000 7f4a44415740 1 Processor -- start 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 0x557eb1f3a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f3a800 0x557eb1f379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557eb1f3d3d0 con 0x557eb1f2ff40 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557eb1e2f0b0 con 0x557eb1f3a800 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557eb1f3d550 con 0x557eb1f2f620 2024-04-25T21:10:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a44415740 10 client.0 _ll_register_callbacks cb 0x557eb1e1cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53324/0 (socket says 172.21.15.37:53324) 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 -- 172.21.15.37:0/2203187249 learned_addr learned my addr 172.21.15.37:0/2203187249 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f3a800 0x557eb1f379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 0x557eb1f3a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 -- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 msgr2=0x557eb1f3a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 0x557eb1f3a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 -- 172.21.15.37:0/2203187249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f3a800 msgr2=0x557eb1f379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f3a800 0x557eb1f379c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 -- 172.21.15.37:0/2203187249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb1c73620 con 0x557eb1f2ff40 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 0x557eb1f3a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a3affd640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00bfc0 tx=0x7f4a2c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1b4270c8e3724f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c004020 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.157+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4a2c0041c0 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a300035f0 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c0043c0 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557eb1f49300 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a300035f0 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4a2c003a60 con 0x557eb1f2ff40 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 20 client.15651 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1f4a810 0x557eb1f4cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 4 client.15651 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb1ea2390 con 0x557eb1f4a810 2024-04-25T21:10:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.158+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1f4a810 0x557eb1f4cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.159+0000 7f4a1eff5640 10 client.15651 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.159+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1f4a810 0x557eb1f4cc00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1abb47e9902a1af server_cookie=8f1cdb32228450cb in_seq=0 out_seq=0 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a1eff5640 1 -- 172.21.15.37:0/2203187249 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eb1ea2390 con 0x557eb1f4a810 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a1eff5640 10 client.15651 handle_command_reply: tid=0 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 client.15651 shutdown 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 msgr2=0x557eb1f37460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00bfc0 tx=0x7f4a2c0118d0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 shutdown_connections 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1f4a810 0x557eb1f4cc00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f3a800 0x557eb1f379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f37460 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a1eff5640 0 client.15651 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 --2- 172.21.15.37:0/2203187249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f2f620 0x557eb1f3a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 >> 172.21.15.37:0/2203187249 conn(0x557eb1f2bb30 msgr2=0x557eb1f31860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 shutdown_connections 2024-04-25T21:10:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 1 -- 172.21.15.37:0/2203187249 wait complete. 2024-04-25T21:10:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.160+0000 7f4a44415740 20 client.15651 trim_cache size 0 max 16384 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 Processor -- start 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 -- start start 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f46290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f39ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1afc80 con 0x557eb1f2ff40 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1b17c0 con 0x557eb1f2f620 2024-04-25T21:10:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a44415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1b0350 con 0x557eb1f3a800 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53330/0 (socket says 172.21.15.37:53330) 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3affd640 1 -- 172.21.15.37:0/184495719 learned_addr learned my addr 172.21.15.37:0/184495719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f46290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3affd640 1 -- 172.21.15.37:0/184495719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 msgr2=0x557eb1f39ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f39ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.176+0000 7f4a3affd640 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f39ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a3affd640 1 -- 172.21.15.37:0/184495719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 msgr2=0x557eb1f46290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a3affd640 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f46290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a3affd640 1 -- 172.21.15.37:0/184495719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb1f36410 con 0x557eb1f2ff40 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a3affd640 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c003dd0 tx=0x7f4a2c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e4639e10ff00db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a20ff9640 1 -- 172.21.15.37:0/184495719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c01a3f0 con 0x557eb1f2ff40 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a20ff9640 1 -- 172.21.15.37:0/184495719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2c01a590 con 0x557eb1f2ff40 2024-04-25T21:10:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.177+0000 7f4a20ff9640 1 -- 172.21.15.37:0/184495719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c005430 con 0x557eb1f2ff40 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- 172.21.15.37:0/184495719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 msgr2=0x557eb1f32970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c003dd0 tx=0x7f4a2c011dd0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- 172.21.15.37:0/184495719 shutdown_connections 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f39ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f32970 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- 172.21.15.37:0/184495719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f46290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- 172.21.15.37:0/184495719 >> 172.21.15.37:0/184495719 conn(0x557eb1f2bb30 msgr2=0x557eb1f32260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- 172.21.15.37:0/184495719 shutdown_connections 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- 172.21.15.37:0/184495719 wait complete. 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- start start 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 Processor -- start 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f42900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f3e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1b10f0 con 0x557eb1f2ff40 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1b0a20 con 0x557eb1f2f620 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1adc60 con 0x557eb1f3a800 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.178+0000 7f4a44415740 10 client.0 _ll_register_callbacks cb 0x7f4a2c00ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53334/0 (socket says 172.21.15.37:53334) 2024-04-25T21:10:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 -- 172.21.15.37:0/3915130095 learned_addr learned my addr 172.21.15.37:0/3915130095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f42900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f3e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 -- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 msgr2=0x557eb1f3e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f3e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 -- 172.21.15.37:0/3915130095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 msgr2=0x557eb1f42900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f42900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 -- 172.21.15.37:0/3915130095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb1f3e860 con 0x557eb1f2ff40 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a3affd640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c003dd0 tx=0x7f4a2c0042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06d17da32c024c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c003940 con 0x557eb1f2ff40 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4a2c003ae0 con 0x557eb1f2ff40 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c01a590 con 0x557eb1f2ff40 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c005040 con 0x557eb1f2ff40 2024-04-25T21:10:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a44415740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.179+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557eb1fb0b40 con 0x557eb1f2ff40 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2c01a590 con 0x557eb1f2ff40 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4a2c01a740 con 0x557eb1f2ff40 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 20 client.15657 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1fb11d0 0x557eb1fb35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 4 client.15657 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557eb1e9c200 con 0x557eb1fb11d0 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.180+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1fb11d0 0x557eb1fb35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.181+0000 7f4a3a7fc640 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1fb11d0 0x557eb1fb35f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25f145b96ea6f49 server_cookie=e21f433e5c75e400 in_seq=0 out_seq=0 2024-04-25T21:10:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.181+0000 7f4a1eff5640 10 client.15657 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.188+0000 7f4a1eff5640 1 -- 172.21.15.37:0/3915130095 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557eb1e9c200 con 0x557eb1fb11d0 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.188+0000 7f4a1eff5640 10 client.15657 handle_command_reply: tid=0 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.188+0000 7f4a44415740 1 client.15657 shutdown 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 msgr2=0x557eb1f3dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c003dd0 tx=0x7f4a2c0042a0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 shutdown_connections 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557eb1fb11d0 0x557eb1fb35f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557eb1f3a800 0x557eb1f3e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557eb1f2ff40 0x557eb1f3dd60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 --2- 172.21.15.37:0/3915130095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557eb1f2f620 0x557eb1f42900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a1eff5640 0 client.15657 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 >> 172.21.15.37:0/3915130095 conn(0x557eb1f2bb30 msgr2=0x557eb1f2acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 shutdown_connections 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 1 -- 172.21.15.37:0/3915130095 wait complete. 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a44415740 20 client.15657 trim_cache size 0 max 16384 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:10:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:10:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 msgr2=0x7f4a240a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 0x7f4a240a62b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4a34066f70 tx=0x7f4a34067440 comp rx=0 tx=0).stop 2024-04-25T21:10:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 msgr2=0x7f4a3c1a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:29.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400eaa0 tx=0x7f4a3400ef60 comp rx=0 tx=0).stop 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 shutdown_connections 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3c074cd0 0x7f4a3c1a6880 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a240a3df0 0x7f4a240a62b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a3c107290 0x7f4a3c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 --2- 172.21.15.37:0/3447536675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a3c1021c0 0x7f4a3c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.189+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 >> 172.21.15.37:0/3447536675 conn(0x7f4a3c0fda20 msgr2=0x7f4a3c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.190+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 shutdown_connections 2024-04-25T21:10:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:29.190+0000 7f4a416fa640 1 -- 172.21.15.37:0/3447536675 wait complete. 2024-04-25T21:10:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:29 smithi177 ceph-mon[33043]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:29 smithi098 ceph-mon[32569]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:29.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:29 smithi037 ceph-mon[29195]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-25T21:10:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:31 smithi177 ceph-mon[33043]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-04-25T21:10:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:31 smithi098 ceph-mon[32569]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-04-25T21:10:31.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:31 smithi037 ceph-mon[29195]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-04-25T21:10:33.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:33 smithi177 ceph-mon[33043]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:10:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:33 smithi098 ceph-mon[32569]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:10:33.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:33 smithi037 ceph-mon[29195]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T21:10:35.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:35 smithi177 ceph-mon[33043]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T21:10:35.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:35 smithi098 ceph-mon[32569]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T21:10:35.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:35 smithi037 ceph-mon[29195]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-25T21:10:36.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:36 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:36.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:36 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:36.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:36 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:36.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:36 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:10:36.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:36 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:36.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:36 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:36.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:36 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:36.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:36 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:10:36.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:36 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:10:36.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:36 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:10:36.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:36 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:10:36.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:36 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:10:37.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:37 smithi177 ceph-mon[33043]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:37.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:37 smithi098 ceph-mon[32569]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:37.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:37 smithi037 ceph-mon[29195]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:39 smithi177 ceph-mon[33043]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:39.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:39 smithi098 ceph-mon[32569]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:39 smithi037 ceph-mon[29195]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:41.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:10:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:41.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:41 smithi037 ceph-mon[29195]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:10:41.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:41 smithi177 ceph-mon[33043]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:10:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:41 smithi098 ceph-mon[32569]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2914401693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b81072a0 msgr2=0x7fd7b810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2914401693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b81072a0 0x7fd7b810ee70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009ca0 tx=0x7fd7b002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2914401693 shutdown_connections 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2914401693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b81072a0 0x7fd7b810ee70 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2914401693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b8102b80 0x7fd7b8106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2914401693 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 0x7fd7b81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2914401693 >> 172.21.15.37:0/2914401693 conn(0x7fd7b80793e0 msgr2=0x7fd7b80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2914401693 shutdown_connections 2024-04-25T21:10:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.281+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2914401693 wait complete. 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 Processor -- start 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 -- start start 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 0x7fd7b8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 0x7fd7b8198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8112210 con 0x7fd7b819cf00 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8112090 con 0x7fd7b81021d0 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bfa10640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8112390 con 0x7fd7b81072a0 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bdfad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 0x7fd7b8198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bdfad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52706/0 (socket says 172.21.15.37:52706) 2024-04-25T21:10:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 0x7fd7b8198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:35040/0 (socket says 172.21.15.37:35040) 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.282+0000 7fd7bdfad640 1 -- 172.21.15.37:0/2481052628 learned_addr learned my addr 172.21.15.37:0/2481052628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 0x7fd7b8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 -- 172.21.15.37:0/2481052628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 msgr2=0x7fd7b8198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 0x7fd7b8198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 -- 172.21.15.37:0/2481052628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 msgr2=0x7fd7b8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 0x7fd7b8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 -- 172.21.15.37:0/2481052628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b819d9f0 con 0x7fd7b819cf00 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00098d0 tx=0x7fd7b00317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7ae7fc640 1 -- 172.21.15.37:0/2481052628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0003b70 con 0x7fd7b819cf00 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7ae7fc640 1 -- 172.21.15.37:0/2481052628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7b0003d10 con 0x7fd7b819cf00 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7ae7fc640 1 -- 172.21.15.37:0/2481052628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b002fa20 con 0x7fd7b819cf00 2024-04-25T21:10:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b819dc80 con 0x7fd7b819cf00 2024-04-25T21:10:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.283+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b81a5450 con 0x7fd7b819cf00 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.285+0000 7fd7ae7fc640 1 -- 172.21.15.37:0/2481052628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7b002fc40 con 0x7fd7b819cf00 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 Processor -- start 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 -- start start 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea685a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7ae7fc640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 0x7fd7a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea692bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561dea5ea970 con 0x561dea685f40 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561dea5874a0 con 0x561dea685620 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7c2802740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561dea5e6930 con 0x561dea690800 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 0x7fd7a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52722/0 (socket says 172.21.15.37:52722) 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bcfab640 1 -- 172.21.15.37:0/4085167914 learned_addr learned my addr 172.21.15.37:0/4085167914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7ae7fc640 1 -- 172.21.15.37:0/2481052628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd7b00f1c00 con 0x7fd7b819cf00 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea685a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.286+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea692bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 -- 172.21.15.37:0/4085167914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 msgr2=0x561dea692bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea692bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 -- 172.21.15.37:0/4085167914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 msgr2=0x561dea685a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea685a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 -- 172.21.15.37:0/4085167914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dea2fd210 con 0x561dea685f40 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 0x7fd7a40a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fd0 tx=0x7fd7b4005850 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800bd40 tx=0x7fd7a8004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7381374e02a8a545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7a37fe640 1 -- 172.21.15.37:0/4085167914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800be30 con 0x561dea685f40 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7a37fe640 1 -- 172.21.15.37:0/4085167914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a80054d0 con 0x561dea685f40 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.287+0000 7fd7a37fe640 1 -- 172.21.15.37:0/4085167914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004df0 con 0x561dea685f40 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- 172.21.15.37:0/4085167914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 msgr2=0x561dea6902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800bd40 tx=0x7fd7a8004130 comp rx=0 tx=0).stop 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- 172.21.15.37:0/4085167914 shutdown_connections 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea692bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6902c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- 172.21.15.37:0/4085167914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea685a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- 172.21.15.37:0/4085167914 >> 172.21.15.37:0/4085167914 conn(0x561dea681b30 msgr2=0x561dea681f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- 172.21.15.37:0/4085167914 shutdown_connections 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- 172.21.15.37:0/4085167914 wait complete. 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- start start 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 Processor -- start 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685f40 0x561dea68d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 0x561dea68da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561dea6933d0 con 0x561dea685620 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561dea5850b0 con 0x561dea690800 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.288+0000 7fd7c2802740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561dea693550 con 0x561dea685f40 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7c2802740 10 client.0 _ll_register_callbacks cb 0x561dea572d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52724/0 (socket says 172.21.15.37:52724) 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 -- 172.21.15.37:0/3461670256 learned_addr learned my addr 172.21.15.37:0/3461670256 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 0x561dea68da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685f40 0x561dea68d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 -- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685f40 msgr2=0x561dea68d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685f40 0x561dea68d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 -- 172.21.15.37:0/3461670256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 msgr2=0x561dea68da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 0x561dea68da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 -- 172.21.15.37:0/3461670256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dea3c9620 con 0x561dea685620 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 0x561dea68da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4011440 tx=0x7fd7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1046a54d5b02c340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4022070 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd7b401e860 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a80054d0 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7b402e050 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7c2802740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.289+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561dea69f3a0 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a80054d0 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd7b4010af0 con 0x561dea685620 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 20 client.15669 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea6a0810 0x561dea6a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 4 client.15669 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dea5f8390 con 0x561dea6a0810 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.290+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea6a0810 0x561dea6a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea6a0810 0x561dea6a2c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b84216d714e0636 server_cookie=3dcdcab52c383ba9 in_seq=0 out_seq=0 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7a0ff9640 10 client.15669 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/3461670256 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561dea5f8390 con 0x561dea6a0810 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7a0ff9640 10 client.15669 handle_command_reply: tid=0 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7c2802740 1 client.15669 shutdown 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 msgr2=0x561dea6901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4011440 tx=0x7fd7b40046c0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 shutdown_connections 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.291+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea6a0810 0x561dea6a2c20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea690800 0x561dea68da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7a0ff9640 0 client.15669 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685f40 0x561dea68d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3461670256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685620 0x561dea6901a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 >> 172.21.15.37:0/3461670256 conn(0x561dea681b30 msgr2=0x561dea6878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 shutdown_connections 2024-04-25T21:10:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 1 -- 172.21.15.37:0/3461670256 wait complete. 2024-04-25T21:10:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.292+0000 7fd7c2802740 20 client.15669 trim_cache size 0 max 16384 2024-04-25T21:10:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 Processor -- start 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 -- start start 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea68fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea6a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ada00 con 0x561dea685f40 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81af540 con 0x561dea685620 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7c2802740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ae0d0 con 0x561dea690800 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52726/0 (socket says 172.21.15.37:52726) 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 -- 172.21.15.37:0/3878751920 learned_addr learned my addr 172.21.15.37:0/3878751920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea68fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 -- 172.21.15.37:0/3878751920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 msgr2=0x561dea6a8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea6a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea6a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 -- 172.21.15.37:0/3878751920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 msgr2=0x561dea68fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea68fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 -- 172.21.15.37:0/3878751920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dea685d70 con 0x561dea685f40 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea68fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.307+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800b620 tx=0x7fd7a800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23477ee04897f43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7a2ffd640 1 -- 172.21.15.37:0/3878751920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a802d020 con 0x561dea685f40 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7a2ffd640 1 -- 172.21.15.37:0/3878751920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a80054d0 con 0x561dea685f40 2024-04-25T21:10:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7a2ffd640 1 -- 172.21.15.37:0/3878751920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8029860 con 0x561dea685f40 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 -- 172.21.15.37:0/3878751920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 msgr2=0x561dea6a8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800b620 tx=0x7fd7a800b5b0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 -- 172.21.15.37:0/3878751920 shutdown_connections 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea690800 0x561dea6a8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea685f40 0x561dea6a8530 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 --2- 172.21.15.37:0/3878751920 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685620 0x561dea68fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 -- 172.21.15.37:0/3878751920 >> 172.21.15.37:0/3878751920 conn(0x561dea681b30 msgr2=0x561dea68d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 -- 172.21.15.37:0/3878751920 shutdown_connections 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.308+0000 7fd7c2802740 1 -- 172.21.15.37:0/3878751920 wait complete. 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 -- start start 2024-04-25T21:10:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 Processor -- start 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685620 0x561dea698a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685f40 0x561dea695da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81aee70 con 0x561dea690800 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ae7a0 con 0x561dea685f40 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b81ab9e0 con 0x561dea685620 2024-04-25T21:10:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7c2802740 10 client.0 _ll_register_callbacks cb 0x7fd7b400fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52740/0 (socket says 172.21.15.37:52740) 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 -- 172.21.15.37:0/1639408989 learned_addr learned my addr 172.21.15.37:0/1639408989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685620 0x561dea698a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bcfab640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685f40 0x561dea695da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 -- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685620 msgr2=0x561dea698a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685620 0x561dea698a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 -- 172.21.15.37:0/1639408989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685f40 msgr2=0x561dea695da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685f40 0x561dea695da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.309+0000 7fd7bdfad640 1 -- 172.21.15.37:0/1639408989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dea696980 con 0x561dea690800 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7bdfad640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009430 tx=0x7fd7b00bd5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d4f0e110d11d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00be380 con 0x561dea690800 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd7b00ef5a0 con 0x561dea690800 2024-04-25T21:10:42.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a80054d0 con 0x561dea690800 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd7b00ef7a0 con 0x561dea690800 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 10 client.15675 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561dea706c40 con 0x561dea690800 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a80054d0 con 0x561dea690800 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd7b00de0b0 con 0x561dea690800 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 20 client.15675 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea7073c0 0x561dea7097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 4 client.15675 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.310+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dea5f2200 con 0x561dea7073c0 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.311+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea7073c0 0x561dea7097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.311+0000 7fd7a0ff9640 10 client.15675 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:42.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.311+0000 7fd7bd7ac640 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea7073c0 0x561dea7097e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba00de0db5e42608 server_cookie=66453b3d7d55496d in_seq=0 out_seq=0 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7a0ff9640 1 -- 172.21.15.37:0/1639408989 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561dea5f2200 con 0x561dea7073c0 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7a0ff9640 10 client.15675 handle_command_reply: tid=0 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 client.15675 shutdown 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 msgr2=0x561dea6962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009430 tx=0x7fd7b00bd5e0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 shutdown_connections 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561dea7073c0 0x561dea7097e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dea690800 0x561dea6962e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7a0ff9640 0 client.15675 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561dea685f40 0x561dea695da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 --2- 172.21.15.37:0/1639408989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561dea685620 0x561dea698a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 >> 172.21.15.37:0/1639408989 conn(0x561dea681b30 msgr2=0x561dea680420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 shutdown_connections 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 1 -- 172.21.15.37:0/1639408989 wait complete. 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7c2802740 20 client.15675 trim_cache size 0 max 16384 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:10:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 msgr2=0x7fd7a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.315+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 0x7fd7a40a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4005fd0 tx=0x7fd7b4005850 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 msgr2=0x7fd7b819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00098d0 tx=0x7fd7b00317b0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 shutdown_connections 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7b819cf00 0x7fd7b819d3b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd7a40a3dc0 0x7fd7a40a6280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7b81072a0 0x7fd7b8198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 --2- 172.21.15.37:0/2481052628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7b81021d0 0x7fd7b8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 >> 172.21.15.37:0/2481052628 conn(0x7fd7b80793e0 msgr2=0x7fd7b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 shutdown_connections 2024-04-25T21:10:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:42.316+0000 7fd7bfa10640 1 -- 172.21.15.37:0/2481052628 wait complete. 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 -- 172.21.15.37:0/1756125651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8107980 msgr2=0x7f90e8107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 --2- 172.21.15.37:0/1756125651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8107980 0x7f90e8107d60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009990 tx=0x7f90e4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 -- 172.21.15.37:0/1756125651 shutdown_connections 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 --2- 172.21.15.37:0/1756125651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 0x7f90e810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 --2- 172.21.15.37:0/1756125651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8107980 0x7f90e8107d60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 --2- 172.21.15.37:0/1756125651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90e8101930 0x7f90e8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 -- 172.21.15.37:0/1756125651 >> 172.21.15.37:0/1756125651 conn(0x7f90e80788b0 msgr2=0x7f90e8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 -- 172.21.15.37:0/1756125651 shutdown_connections 2024-04-25T21:10:43.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.268+0000 7f90efbd1700 1 -- 172.21.15.37:0/1756125651 wait complete. 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 Processor -- start 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 -- start start 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 0x7f90e819b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90e8107980 0x7f90e819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8111b70 con 0x7f90e8101930 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81119f0 con 0x7f90e81022f0 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90efbd1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8111cf0 con 0x7f90e8107980 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90ed96d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90ed96d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52756/0 (socket says 172.21.15.37:52756) 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90ed96d700 1 -- 172.21.15.37:0/2253956369 learned_addr learned my addr 172.21.15.37:0/2253956369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.269+0000 7f90ed16c700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 0x7f90e819b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:43.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 -- 172.21.15.37:0/2253956369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90e8107980 msgr2=0x7f90e819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90e8107980 0x7f90e819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 -- 172.21.15.37:0/2253956369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 msgr2=0x7f90e819b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 0x7f90e819b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 -- 172.21.15.37:0/2253956369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e81a03b0 con 0x7f90e8101930 2024-04-25T21:10:43.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.270+0000 7f90ed96d700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90e4005f40 tx=0x7f90e4009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.271+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4027070 con 0x7f90e8101930 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.271+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e4003fb0 con 0x7f90e8101930 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.271+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e81a0640 con 0x7f90e8101930 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.271+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4022c90 con 0x7f90e8101930 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.271+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e81a7f20 con 0x7f90e8101930 2024-04-25T21:10:43.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.273+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e8069730 con 0x7f90e8101930 2024-04-25T21:10:43.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.275+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90e402a070 con 0x7f90e8101930 2024-04-25T21:10:43.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.275+0000 7f90deffd700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 0x7f90d40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:43.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.275+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f90e40d1d80 con 0x7f90e8101930 2024-04-25T21:10:43.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.275+0000 7f90ed16c700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 0x7f90d40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:43.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.276+0000 7f90ed16c700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 0x7f90d40a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90e8078cd0 tx=0x7f90d8009190 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:43.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.279+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90e4031020 con 0x7f90e8101930 2024-04-25T21:10:43.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:43 smithi037 ceph-mon[29195]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:43.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.541+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90e81a80f0 con 0x7f90e8101930 2024-04-25T21:10:43.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.542+0000 7f90deffd700 1 -- 172.21.15.37:0/2253956369 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f90e4013070 con 0x7f90e8101930 2024-04-25T21:10:43.542 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:10:43.542 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:10:43.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 msgr2=0x7f90d40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 0x7f90d40a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f90e8078cd0 tx=0x7f90d8009190 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 msgr2=0x7f90e819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f90e4005f40 tx=0x7f90e4009ee0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 shutdown_connections 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90d40a3e30 0x7f90d40a62f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90e8107980 0x7f90e819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90e8101930 0x7f90e819b3a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 --2- 172.21.15.37:0/2253956369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90e81022f0 0x7f90e819b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 >> 172.21.15.37:0/2253956369 conn(0x7f90e80788b0 msgr2=0x7f90e806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 shutdown_connections 2024-04-25T21:10:43.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:43.544+0000 7f90efbd1700 1 -- 172.21.15.37:0/2253956369 wait complete. 2024-04-25T21:10:43.546 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:10:43.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:43 smithi177 ceph-mon[33043]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:43 smithi098 ceph-mon[32569]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T21:10:44.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id bb93a79b --release --await' 2024-04-25T21:10:44.371 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id bb93a79b --release --await 2024-04-25T21:10:44.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:44 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2253956369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:44.663 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:10:44.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:44 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2253956369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:44.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:44 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2253956369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:10:45.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:45 smithi037 ceph-mon[29195]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T21:10:45.568 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:45 smithi098 ceph-mon[32569]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T21:10:45.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:45 smithi177 ceph-mon[33043]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-25T21:10:46.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 -- 172.21.15.37:0/3172651617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198102a90 msgr2=0x7fd198102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 --2- 172.21.15.37:0/3172651617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198102a90 0x7fd198102f10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009990 tx=0x7fd18c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 -- 172.21.15.37:0/3172651617 shutdown_connections 2024-04-25T21:10:46.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 --2- 172.21.15.37:0/3172651617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 0x7fd19810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 --2- 172.21.15.37:0/3172651617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198108a50 0x7fd198108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 --2- 172.21.15.37:0/3172651617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198102a90 0x7fd198102f10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 -- 172.21.15.37:0/3172651617 >> 172.21.15.37:0/3172651617 conn(0x7fd198079ba0 msgr2=0x7fd198079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 -- 172.21.15.37:0/3172651617 shutdown_connections 2024-04-25T21:10:46.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 -- 172.21.15.37:0/3172651617 wait complete. 2024-04-25T21:10:46.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.515+0000 7fd19dd86700 1 Processor -- start 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 -- start start 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198102a90 0x7fd198074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 0x7fd198075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112c20 con 0x7fd198108a50 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112aa0 con 0x7fd198102a90 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd19dd86700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd198112da0 con 0x7fd198103450 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52772/0 (socket says 172.21.15.37:52772) 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 -- 172.21.15.37:0/4218129506 learned_addr learned my addr 172.21.15.37:0/4218129506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd196ffd700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 0x7fd198075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198102a90 0x7fd198074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 -- 172.21.15.37:0/4218129506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 msgr2=0x7fd198075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 0x7fd198075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 -- 172.21.15.37:0/4218129506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198102a90 msgr2=0x7fd198074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198102a90 0x7fd198074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.516+0000 7fd197fff700 1 -- 172.21.15.37:0/4218129506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981a76a0 con 0x7fd198108a50 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd197fff700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd19000ada0 tx=0x7fd19000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd194ff9700 1 -- 172.21.15.37:0/4218129506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017020 con 0x7fd198108a50 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd194ff9700 1 -- 172.21.15.37:0/4218129506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19000ce30 con 0x7fd198108a50 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981a7990 con 0x7fd198108a50 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd194ff9700 1 -- 172.21.15.37:0/4218129506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190005350 con 0x7fd198108a50 2024-04-25T21:10:46.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.517+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981a7ea0 con 0x7fd198108a50 2024-04-25T21:10:46.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.519+0000 7fd194ff9700 1 -- 172.21.15.37:0/4218129506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd19000c980 con 0x7fd198108a50 2024-04-25T21:10:46.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.520+0000 7fd194ff9700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 0x7fd1800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.520+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 0x7fd1800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.520+0000 7fd194ff9700 1 -- 172.21.15.37:0/4218129506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd1900cff90 con 0x7fd198108a50 2024-04-25T21:10:46.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.520+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 0x7fd1800a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd188005fd0 tx=0x7fd188005db0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 Processor -- start 2024-04-25T21:10:46.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 -- start start 2024-04-25T21:10:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a54650 0x5606d7a54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 0x5606d7a62260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5606d79a7d90 con 0x5606d7a54f70 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5606d794ea30 con 0x5606d7a5fe70 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5606d79a3d50 con 0x5606d7a54650 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52776/0 (socket says 172.21.15.37:52776) 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 -- 172.21.15.37:0/3341655828 learned_addr learned my addr 172.21.15.37:0/3341655828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd1977fe700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a54650 0x5606d7a54a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd197fff700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 0x5606d7a62260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 -- 172.21.15.37:0/3341655828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a54650 msgr2=0x5606d7a54a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a54650 0x5606d7a54a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 -- 172.21.15.37:0/3341655828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 msgr2=0x5606d7a62260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 0x5606d7a62260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd196ffd700 1 -- 172.21.15.37:0/3341655828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d7912840 con 0x5606d7a54f70 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.521+0000 7fd197fff700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 0x5606d7a62260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd196ffd700 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009990 tx=0x7fd18c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33abb431ccd864c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd17dffb700 1 -- 172.21.15.37:0/3341655828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c003a60 con 0x5606d7a54f70 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd17dffb700 1 -- 172.21.15.37:0/3341655828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18c026070 con 0x5606d7a54f70 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd17dffb700 1 -- 172.21.15.37:0/3341655828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c005470 con 0x5606d7a54f70 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd1a860fb80 1 -- 172.21.15.37:0/3341655828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 msgr2=0x5606d7a5f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd18c009990 tx=0x7fd18c003680 comp rx=0 tx=0).stop 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd1a860fb80 1 -- 172.21.15.37:0/3341655828 shutdown_connections 2024-04-25T21:10:46.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.522+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a5fe70 0x5606d7a62260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a54650 0x5606d7a54a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/3341655828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5f930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 -- 172.21.15.37:0/3341655828 >> 172.21.15.37:0/3341655828 conn(0x5606d7981990 msgr2=0x5606d7a50b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 -- 172.21.15.37:0/3341655828 shutdown_connections 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 -- 172.21.15.37:0/3341655828 wait complete. 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 -- start start 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 Processor -- start 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a5f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.523+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5606d7a628a0 con 0x5606d7a54f70 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5606d79616b0 con 0x5606d7a54650 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5606d7a62a20 con 0x5606d7a5fe70 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd1a860fb80 10 client.0 _ll_register_callbacks cb 0x5606d7940a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52780/0 (socket says 172.21.15.37:52780) 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 -- 172.21.15.37:0/5102485 learned_addr learned my addr 172.21.15.37:0/5102485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd1977fe700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a5f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd197fff700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 -- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 msgr2=0x5606d7a5ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 -- 172.21.15.37:0/5102485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 msgr2=0x5606d7a5f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a5f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 -- 172.21.15.37:0/5102485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d78f59d0 con 0x5606d7a54f70 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd196ffd700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005f10 tx=0x7fd18c005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1a899b6819193a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd176ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ba50 con 0x5606d7a54f70 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd18c00bbf0 con 0x5606d7a54f70 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.524+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c026070 con 0x5606d7a54f70 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd18c00bdf0 con 0x5606d7a54f70 2024-04-25T21:10:46.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 10 client.15693 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606d7a6d0e0 con 0x5606d7a54f70 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18c00f910 con 0x5606d7a54f70 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd18c032400 con 0x5606d7a54f70 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 10 client.15693 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 20 client.15693 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7a6f5b0 0x5606d7a719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 4 client.15693 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd18c026070 con 0x5606d7a6f5b0 2024-04-25T21:10:46.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.525+0000 7fd1977fe700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7a6f5b0 0x5606d7a719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.526+0000 7fd1977fe700 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7a6f5b0 0x5606d7a719a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2d66470c7614e3d server_cookie=1c84e6c4c081e2da in_seq=0 out_seq=0 2024-04-25T21:10:46.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.526+0000 7fd176ffd700 10 client.15693 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:46.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd176ffd700 1 -- 172.21.15.37:0/5102485 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fd18c026070 con 0x5606d7a6f5b0 2024-04-25T21:10:46.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd176ffd700 10 client.15693 handle_command_reply: tid=0 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 client.15693 shutdown 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 msgr2=0x5606d7a5c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005f10 tx=0x7fd18c005340 comp rx=0 tx=0).stop 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 shutdown_connections 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7a6f5b0 0x5606d7a719a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a5f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/5102485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a5c910 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 >> 172.21.15.37:0/5102485 conn(0x5606d7981990 msgr2=0x5606d7a56ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd176ffd700 0 client.15693 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 shutdown_connections 2024-04-25T21:10:46.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 1 -- 172.21.15.37:0/5102485 wait complete. 2024-04-25T21:10:46.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.527+0000 7fd1a860fb80 20 client.15693 trim_cache size 0 max 16384 2024-04-25T21:10:46.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.546+0000 7fd1a860fb80 1 Processor -- start 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.546+0000 7fd1a860fb80 1 -- start start 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54f70 0x5606d7a50c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981af420 con 0x5606d7a54650 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b0f60 con 0x5606d7a54f70 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981afaf0 con 0x5606d7a5fe70 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52786/0 (socket says 172.21.15.37:52786) 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 -- 172.21.15.37:0/47185427 learned_addr learned my addr 172.21.15.37:0/47185427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd197fff700 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 -- 172.21.15.37:0/47185427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 msgr2=0x5606d7a5f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 -- 172.21.15.37:0/47185427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54f70 msgr2=0x5606d7a50c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54f70 0x5606d7a50c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 -- 172.21.15.37:0/47185427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606d7a5d810 con 0x5606d7a54650 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd1977fe700 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd188005e20 tx=0x7fd188014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee640292163d3ade server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd17d7fa700 1 -- 172.21.15.37:0/47185427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18801faa0 con 0x5606d7a54650 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.547+0000 7fd17d7fa700 1 -- 172.21.15.37:0/47185427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18802b030 con 0x5606d7a54650 2024-04-25T21:10:46.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd17d7fa700 1 -- 172.21.15.37:0/47185427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188004b20 con 0x5606d7a54650 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- 172.21.15.37:0/47185427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 msgr2=0x5606d7a5b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd188005e20 tx=0x7fd188014000 comp rx=0 tx=0).stop 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- 172.21.15.37:0/47185427 shutdown_connections 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a5f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54650 0x5606d7a5b8d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/47185427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54f70 0x5606d7a50c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- 172.21.15.37:0/47185427 >> 172.21.15.37:0/47185427 conn(0x5606d7981990 msgr2=0x5606d7a6acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- 172.21.15.37:0/47185427 shutdown_connections 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- 172.21.15.37:0/47185427 wait complete. 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.548+0000 7fd1a860fb80 1 -- start start 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 Processor -- start 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a72a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b0890 con 0x5606d7a54f70 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981b01c0 con 0x5606d7a54650 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1981ad3c0 con 0x5606d7a5fe70 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1a860fb80 10 client.0 _ll_register_callbacks cb 0x5606d7940a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd196ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52790/0 (socket says 172.21.15.37:52790) 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd196ffd700 1 -- 172.21.15.37:0/4280404376 learned_addr learned my addr 172.21.15.37:0/4280404376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd197fff700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a72a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.549+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 -- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 msgr2=0x5606d7a72a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a72a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 -- 172.21.15.37:0/4280404376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 msgr2=0x5606d7a72010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a72010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 -- 172.21.15.37:0/4280404376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606d7ad47c0 con 0x5606d7a54f70 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a72010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd196ffd700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd18c032850 tx=0x7fd18c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a0547eb9abb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c032c50 con 0x5606d7a54f70 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd18c00b9d0 con 0x5606d7a54f70 2024-04-25T21:10:46.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18802b030 con 0x5606d7a54f70 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd18c005460 con 0x5606d7a54f70 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1a860fb80 10 client.15699 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5606d7ad5ac0 con 0x5606d7a54f70 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd18802b030 con 0x5606d7a54f70 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd18c02a3f0 con 0x5606d7a54f70 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1a860fb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.550+0000 7fd1a860fb80 10 client.15699 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1a860fb80 20 client.15699 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7ad6140 0x5606d7ad8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1a860fb80 4 client.15699 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bb93a79b", "release": true, "await": true}] 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bb93a79b", "release": true, "await": true}) -- 0x5606d79b5580 con 0x5606d7ad6140 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7ad6140 0x5606d7ad8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd1977fe700 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7ad6140 0x5606d7ad8560 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f19341b830c32b server_cookie=ad738d2244a1f0ef in_seq=0 out_seq=0 2024-04-25T21:10:46.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.551+0000 7fd176ffd700 10 client.15699 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:46.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd176ffd700 1 -- 172.21.15.37:0/4280404376 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5606d79b5580 con 0x5606d7ad6140 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd176ffd700 10 client.15699 handle_command_reply: tid=0 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 client.15699 shutdown 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 msgr2=0x5606d7a72550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd18c032850 tx=0x7fd18c009890 comp rx=0 tx=0).stop 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 shutdown_connections 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5606d7ad6140 0x5606d7ad8560 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5606d7a5fe70 0x5606d7a72a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5606d7a54650 0x5606d7a72010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 --2- 172.21.15.37:0/4280404376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5606d7a54f70 0x5606d7a72550 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 >> 172.21.15.37:0/4280404376 conn(0x5606d7981990 msgr2=0x5606d7a6b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd176ffd700 0 client.15699 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 shutdown_connections 2024-04-25T21:10:46.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 1 -- 172.21.15.37:0/4280404376 wait complete. 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.555+0000 7fd1a860fb80 20 client.15699 trim_cache size 0 max 16384 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "bb93a79b": { 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-04-25T21:10:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.2, 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:10:46.562 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:10:46.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 msgr2=0x7fd1800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 0x7fd1800a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd188005fd0 tx=0x7fd188005db0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 msgr2=0x7fd1980757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd19000ada0 tx=0x7fd19000c330 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 shutdown_connections 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1800a3db0 0x7fd1800a6270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd198108a50 0x7fd1980757c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd198102a90 0x7fd198074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 --2- 172.21.15.37:0/4218129506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd198103450 0x7fd198075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 >> 172.21.15.37:0/4218129506 conn(0x7fd198079ba0 msgr2=0x7fd1981017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 shutdown_connections 2024-04-25T21:10:46.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:10:46.556+0000 7fd19dd86700 1 -- 172.21.15.37:0/4218129506 wait complete. 2024-04-25T21:10:47.259 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bb93a79b', seconds elapsed: 38.4 2024-04-25T21:10:47.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:47 smithi037 ceph-mon[29195]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:47.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:47 smithi177 ceph-mon[33043]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:47.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:47 smithi098 ceph-mon[32569]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:49.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:49 smithi177 ceph-mon[33043]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:49.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:49 smithi098 ceph-mon[32569]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:49.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:49 smithi037 ceph-mon[29195]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-25T21:10:51.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:51 smithi037 ceph-mon[29195]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-04-25T21:10:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:51 smithi177 ceph-mon[33043]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-04-25T21:10:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:51 smithi098 ceph-mon[32569]: pgmap v74: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 KiB/s wr, 14 op/s 2024-04-25T21:10:53.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:53 smithi177 ceph-mon[33043]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-04-25T21:10:53.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:53 smithi098 ceph-mon[32569]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-04-25T21:10:53.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:53 smithi037 ceph-mon[29195]: pgmap v75: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2024-04-25T21:10:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 -- 172.21.15.37:0/3419054991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780101aa0 msgr2=0x7f9780105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 --2- 172.21.15.37:0/3419054991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780101aa0 0x7f9780105a90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002a410 comp rx=0 tx=0).stop 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 -- 172.21.15.37:0/3419054991 shutdown_connections 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 --2- 172.21.15.37:0/3419054991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9780106170 0x7f978010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 --2- 172.21.15.37:0/3419054991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9780101aa0 0x7f9780105a90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 --2- 172.21.15.37:0/3419054991 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97801010f0 0x7f97801014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 -- 172.21.15.37:0/3419054991 >> 172.21.15.37:0/3419054991 conn(0x7f9780077fd0 msgr2=0x7f97800783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 -- 172.21.15.37:0/3419054991 shutdown_connections 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.415+0000 7f97879cc640 1 -- 172.21.15.37:0/3419054991 wait complete. 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 Processor -- start 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 -- start start 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9780106170 0x7f9780198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 0x7f978019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9780111090 con 0x7f97801010f0 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9780110f10 con 0x7f9780106170 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f97879cc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9780111210 con 0x7f978019cf60 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f9785768640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.416+0000 7f9785768640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52934/0 (socket says 172.21.15.37:52934) 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 -- 172.21.15.37:0/2413450661 learned_addr learned my addr 172.21.15.37:0/2413450661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9784f67640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9780106170 0x7f9780198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785f69640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 0x7f978019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 -- 172.21.15.37:0/2413450661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 msgr2=0x7f978019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 0x7f978019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 -- 172.21.15.37:0/2413450661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9780106170 msgr2=0x7f9780198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9780106170 0x7f9780198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 -- 172.21.15.37:0/2413450661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978019da50 con 0x7f97801010f0 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785f69640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 0x7f978019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f9785768640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f977c00d950 tx=0x7f977c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f97767fc640 1 -- 172.21.15.37:0/2413450661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c017070 con 0x7f97801010f0 2024-04-25T21:10:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978019dd40 con 0x7f97801010f0 2024-04-25T21:10:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f97767fc640 1 -- 172.21.15.37:0/2413450661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f977c009c30 con 0x7f97801010f0 2024-04-25T21:10:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f97767fc640 1 -- 172.21.15.37:0/2413450661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c00f620 con 0x7f97801010f0 2024-04-25T21:10:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.417+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a5500 con 0x7f97801010f0 2024-04-25T21:10:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f97767fc640 1 -- 172.21.15.37:0/2413450661 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f977c010460 con 0x7f97801010f0 2024-04-25T21:10:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 Processor -- start 2024-04-25T21:10:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 -- start start 2024-04-25T21:10:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd7002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd65a970 con 0x55c6cd700800 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd5f74a0 con 0x55c6cd6f5f40 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f978a7b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd656930 con 0x55c6cd6f5620 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f9784f67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f9785f69640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f9785f69640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:58670/0 (socket says 172.21.15.37:58670) 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f9784f67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52944/0 (socket says 172.21.15.37:52944) 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.420+0000 7f9785f69640 1 -- 172.21.15.37:0/897900719 learned_addr learned my addr 172.21.15.37:0/897900719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f97767fc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 0x7f976c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9785768640 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd7002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9785f69640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 0x7f976c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 -- 172.21.15.37:0/897900719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 msgr2=0x55c6cd6f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 -- 172.21.15.37:0/897900719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 msgr2=0x55c6cd7002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd7002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 -- 172.21.15.37:0/897900719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cd36d210 con 0x55c6cd700800 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f97767fc640 1 -- 172.21.15.37:0/2413450661 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f977c0d0410 con 0x7f97801010f0 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9784f67640 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9770006010 tx=0x7f9770005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210eb979972b6ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f9785f69640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 0x7f976c0a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9778014950 tx=0x7f9778004ab0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f976b7fe640 1 -- 172.21.15.37:0/897900719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770040070 con 0x55c6cd700800 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f976b7fe640 1 -- 172.21.15.37:0/897900719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9770004030 con 0x55c6cd700800 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.421+0000 7f976b7fe640 1 -- 172.21.15.37:0/897900719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770035040 con 0x55c6cd700800 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 -- 172.21.15.37:0/897900719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 msgr2=0x55c6cd702bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9770006010 tx=0x7f9770005c00 comp rx=0 tx=0).stop 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 -- 172.21.15.37:0/897900719 shutdown_connections 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd702bf0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd7002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 --2- 172.21.15.37:0/897900719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 -- 172.21.15.37:0/897900719 >> 172.21.15.37:0/897900719 conn(0x55c6cd6f1b30 msgr2=0x55c6cd6f1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 -- 172.21.15.37:0/897900719 shutdown_connections 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.422+0000 7f978a7b7740 1 -- 172.21.15.37:0/897900719 wait complete. 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 -- start start 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 Processor -- start 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd700190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd700800 0x55c6cd6fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd7033d0 con 0x55c6cd6f5f40 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd5f50b0 con 0x55c6cd700800 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c6cd703550 con 0x55c6cd6f5620 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f978a7b7740 10 client.0 _ll_register_callbacks cb 0x55c6cd5e2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9784f67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9784f67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52948/0 (socket says 172.21.15.37:52948) 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9784f67640 1 -- 172.21.15.37:0/3367117671 learned_addr learned my addr 172.21.15.37:0/3367117671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9785768640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd700190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9784f67640 1 -- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 msgr2=0x55c6cd700190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.423+0000 7f9785f69640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd700800 0x55c6cd6fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9784f67640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd700190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9784f67640 1 -- 172.21.15.37:0/3367117671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd700800 msgr2=0x55c6cd6fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9784f67640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd700800 0x55c6cd6fda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9784f67640 1 -- 172.21.15.37:0/3367117671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cd439620 con 0x55c6cd6f5f40 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9784f67640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f97700095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38f367110454b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:55.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977004c070 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9770005460 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9770004030 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9770058050 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f978a7b7740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6cd70f320 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.424+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9770004030 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9770004c90 con 0x55c6cd6f5f40 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 20 client.15711 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd710670 0x55c6cd712a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 4 client.15711 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cd668390 con 0x55c6cd710670 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f9785768640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd710670 0x55c6cd712a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f9768ff9640 10 client.15711 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.425+0000 7f9785768640 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd710670 0x55c6cd712a60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989413bd60ea1554 server_cookie=d301bc79e1f65232 in_seq=0 out_seq=0 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f9768ff9640 1 -- 172.21.15.37:0/3367117671 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6cd668390 con 0x55c6cd710670 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f9768ff9640 10 client.15711 handle_command_reply: tid=0 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 client.15711 shutdown 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 msgr2=0x55c6cd6fd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f97700095a0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 shutdown_connections 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd710670 0x55c6cd712a60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd700800 0x55c6cd6fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6fd480 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3367117671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5620 0x55c6cd700190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 >> 172.21.15.37:0/3367117671 conn(0x55c6cd6f1b30 msgr2=0x55c6cd6f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f9768ff9640 0 client.15711 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 shutdown_connections 2024-04-25T21:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 1 -- 172.21.15.37:0/3367117671 wait complete. 2024-04-25T21:10:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.426+0000 7f978a7b7740 20 client.15711 trim_cache size 0 max 16384 2024-04-25T21:10:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.441+0000 7f978a7b7740 1 Processor -- start 2024-04-25T21:10:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.441+0000 7f978a7b7740 1 -- start start 2024-04-25T21:10:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5620 0x55c6cd70c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5f40 0x55c6cd708b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97801ada40 con 0x55c6cd700800 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97801af580 con 0x55c6cd6f5620 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f978a7b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97801ae110 con 0x55c6cd6f5f40 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52956/0 (socket says 172.21.15.37:52956) 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 -- 172.21.15.37:0/3001600012 learned_addr learned my addr 172.21.15.37:0/3001600012 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9784f67640 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5f40 0x55c6cd708b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 -- 172.21.15.37:0/3001600012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5f40 msgr2=0x55c6cd708b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785768640 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5620 0x55c6cd70c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5f40 0x55c6cd708b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 -- 172.21.15.37:0/3001600012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5620 msgr2=0x55c6cd70c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5620 0x55c6cd70c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 -- 172.21.15.37:0/3001600012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cd6f7fe0 con 0x55c6cd700800 2024-04-25T21:10:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f9785f69640 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9778005c20 tx=0x7f9778009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4c10b48e8c95c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f976affd640 1 -- 172.21.15.37:0/3001600012 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977801a070 con 0x55c6cd700800 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f976affd640 1 -- 172.21.15.37:0/3001600012 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778025030 con 0x55c6cd700800 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.442+0000 7f976affd640 1 -- 172.21.15.37:0/3001600012 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778013930 con 0x55c6cd700800 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- 172.21.15.37:0/3001600012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 msgr2=0x55c6cd6ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9778005c20 tx=0x7f9778009920 comp rx=0 tx=0).stop 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- 172.21.15.37:0/3001600012 shutdown_connections 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd700800 0x55c6cd6ffcc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd6f5f40 0x55c6cd708b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 --2- 172.21.15.37:0/3001600012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5620 0x55c6cd70c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- 172.21.15.37:0/3001600012 >> 172.21.15.37:0/3001600012 conn(0x55c6cd6f1b30 msgr2=0x55c6cd6f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- 172.21.15.37:0/3001600012 shutdown_connections 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- 172.21.15.37:0/3001600012 wait complete. 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 -- start start 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.443+0000 7f978a7b7740 1 Processor -- start 2024-04-25T21:10:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6f9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 0x55c6cd6fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97801aeeb0 con 0x55c6cd6f5620 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97801ae7e0 con 0x55c6cd6f5f40 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f97801aba20 con 0x55c6cd700800 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f978a7b7740 10 client.0 _ll_register_callbacks cb 0x7f9770032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52972/0 (socket says 172.21.15.37:52972) 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 -- 172.21.15.37:0/164166914 learned_addr learned my addr 172.21.15.37:0/164166914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9784f67640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6f9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785f69640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 0x55c6cd6fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 -- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 msgr2=0x55c6cd6fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 0x55c6cd6fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 -- 172.21.15.37:0/164166914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 msgr2=0x55c6cd6f9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:10:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6f9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785768640 1 -- 172.21.15.37:0/164166914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cd6fa9c0 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.444+0000 7f9785f69640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 0x55c6cd6fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9785768640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f977c021000 tx=0x7f977c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07058cf9def305d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c09c3e0 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f977c098a50 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9778025030 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f977c0a0790 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6cd77aee0 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778025030 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f977c098130 con 0x55c6cd6f5620 2024-04-25T21:10:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 20 client.15717 populate_metadata read hostname 'smithi037' 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd77b600 0x55c6cd77da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 4 client.15717 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6cd662200 con 0x55c6cd77b600 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.445+0000 7f9784f67640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd77b600 0x55c6cd77da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.446+0000 7f9768ff9640 10 client.15717 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.446+0000 7f9784f67640 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd77b600 0x55c6cd77da20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=727e1ca3a11daf72 server_cookie=c557326caa0573c6 in_seq=0 out_seq=0 2024-04-25T21:10:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.454+0000 7f9768ff9640 1 -- 172.21.15.37:0/164166914 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6cd662200 con 0x55c6cd77b600 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.454+0000 7f9768ff9640 10 client.15717 handle_command_reply: tid=0 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.454+0000 7f978a7b7740 1 client.15717 shutdown 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 msgr2=0x55c6cd6f9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f977c021000 tx=0x7f977c010cb0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 shutdown_connections 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c6cd77b600 0x55c6cd77da20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c6cd700800 0x55c6cd6fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c6cd6f5f40 0x55c6cd6f9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f9768ff9640 0 client.15717 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 --2- 172.21.15.37:0/164166914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c6cd6f5620 0x55c6cd6f9710 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 >> 172.21.15.37:0/164166914 conn(0x55c6cd6f1b30 msgr2=0x55c6cd6f6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 shutdown_connections 2024-04-25T21:10:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 1 -- 172.21.15.37:0/164166914 wait complete. 2024-04-25T21:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f978a7b7740 20 client.15717 trim_cache size 0 max 16384 2024-04-25T21:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:10:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 msgr2=0x7f976c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 0x7f976c0a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9778014950 tx=0x7f9778004ab0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 msgr2=0x7f9780198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f977c00d950 tx=0x7f977c00de10 comp rx=0 tx=0).stop 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 shutdown_connections 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f976c0a3dc0 0x7f976c0a6280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f978019cf60 0x7f978019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.455+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9780106170 0x7f9780198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.456+0000 7f97879cc640 1 --2- 172.21.15.37:0/2413450661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97801010f0 0x7f9780198740 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.456+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 >> 172.21.15.37:0/2413450661 conn(0x7f9780077fd0 msgr2=0x7f9780104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.456+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 shutdown_connections 2024-04-25T21:10:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:10:55.456+0000 7f97879cc640 1 -- 172.21.15.37:0/2413450661 wait complete. 2024-04-25T21:10:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:55 smithi177 ceph-mon[33043]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T21:10:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:55 smithi098 ceph-mon[32569]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T21:10:55.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:55 smithi037 ceph-mon[29195]: pgmap v76: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-25T21:10:57.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:57 smithi177 ceph-mon[33043]: pgmap v77: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:10:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:57 smithi098 ceph-mon[32569]: pgmap v77: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:10:57.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:57 smithi037 ceph-mon[29195]: pgmap v77: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:10:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:10:59 smithi177 ceph-mon[33043]: pgmap v78: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:10:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:10:59 smithi098 ceph-mon[32569]: pgmap v78: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:10:59.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:10:59 smithi037 ceph-mon[29195]: pgmap v78: 97 pgs: 97 active+clean; 1.6 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:11:01.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:01 smithi177 ceph-mon[33043]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:11:01.592 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:01 smithi037 ceph-mon[29195]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:11:01.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:01 smithi098 ceph-mon[32569]: pgmap v79: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T21:11:03.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:03 smithi177 ceph-mon[33043]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T21:11:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:03 smithi098 ceph-mon[32569]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T21:11:03.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:03 smithi037 ceph-mon[29195]: pgmap v80: 97 pgs: 97 active+clean; 1.7 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-25T21:11:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:05 smithi177 ceph-mon[33043]: pgmap v81: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T21:11:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:05 smithi098 ceph-mon[32569]: pgmap v81: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T21:11:05.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:05 smithi037 ceph-mon[29195]: pgmap v81: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 51 KiB/s wr, 36 op/s 2024-04-25T21:11:07.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:07 smithi177 ceph-mon[33043]: pgmap v82: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:07 smithi098 ceph-mon[32569]: pgmap v82: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:07.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:07 smithi037 ceph-mon[29195]: pgmap v82: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 -- 172.21.15.37:0/3834392463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27181021c0 msgr2=0x7f27181025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 --2- 172.21.15.37:0/3834392463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27181021c0 0x7f27181025a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2708009ca0 tx=0x7f270802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 -- 172.21.15.37:0/3834392463 shutdown_connections 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 --2- 172.21.15.37:0/3834392463 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2718107290 0x7f271810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 --2- 172.21.15.37:0/3834392463 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718102b70 0x7f2718106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 --2- 172.21.15.37:0/3834392463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27181021c0 0x7f27181025a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 -- 172.21.15.37:0/3834392463 >> 172.21.15.37:0/3834392463 conn(0x7f27180fda20 msgr2=0x7f27180ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 -- 172.21.15.37:0/3834392463 shutdown_connections 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 -- 172.21.15.37:0/3834392463 wait complete. 2024-04-25T21:11:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.556+0000 7f271d29e640 1 Processor -- start 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 -- start start 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 0x7f27181986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718107290 0x7f2718198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2718112200 con 0x7f271819cf80 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2718112080 con 0x7f2718107290 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f271d29e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2718112380 con 0x7f27181021c0 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47450/0 (socket says 172.21.15.37:47450) 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 -- 172.21.15.37:0/3017001353 learned_addr learned my addr 172.21.15.37:0/3017001353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27167fc640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718107290 0x7f2718198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 0x7f27181986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 -- 172.21.15.37:0/3017001353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 msgr2=0x7f27181986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 0x7f27181986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 -- 172.21.15.37:0/3017001353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718107290 msgr2=0x7f2718198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718107290 0x7f2718198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f27177fe640 1 -- 172.21.15.37:0/3017001353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271819da70 con 0x7f271819cf80 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.557+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 0x7f27181986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f27177fe640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f271000e970 tx=0x7f271000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f26fffff640 1 -- 172.21.15.37:0/3017001353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271000ccc0 con 0x7f271819cf80 2024-04-25T21:11:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271819dd60 con 0x7f271819cf80 2024-04-25T21:11:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f26fffff640 1 -- 172.21.15.37:0/3017001353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f271000ce60 con 0x7f271819cf80 2024-04-25T21:11:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f26fffff640 1 -- 172.21.15.37:0/3017001353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710010430 con 0x7f271819cf80 2024-04-25T21:11:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.558+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a55a0 con 0x7f271819cf80 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.560+0000 7f26fffff640 1 -- 172.21.15.37:0/3017001353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2710004f90 con 0x7f271819cf80 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 Processor -- start 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 -- start start 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735116a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x564735123bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56473507b970 con 0x564735116f40 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5647350184a0 con 0x564735121800 2024-04-25T21:11:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f271ffb0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564735077930 con 0x564735116620 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f26fffff640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 0x7f27000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47452/0 (socket says 172.21.15.37:47452) 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 -- 172.21.15.37:0/3803761710 learned_addr learned my addr 172.21.15.37:0/3803761710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f27177fe640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 0x7f27000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f27167fc640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735116a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f27167fc640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x564735123bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 -- 172.21.15.37:0/3803761710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 msgr2=0x564735116a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f26fffff640 1 -- 172.21.15.37:0/3017001353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f27100d0bb0 con 0x7f271819cf80 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735116a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 -- 172.21.15.37:0/3803761710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 msgr2=0x564735123bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x564735123bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f2716ffd640 1 -- 172.21.15.37:0/3803761710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564734d8e210 con 0x564735116f40 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.561+0000 7f27167fc640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x564735123bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.562+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2708035000 tx=0x7f27080093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ddad1ae9aa3d95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.562+0000 7f27177fe640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 0x7f27000a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2710098fe0 tx=0x7f2710067620 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.562+0000 7f26fd7fa640 1 -- 172.21.15.37:0/3803761710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708031cf0 con 0x564735116f40 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.562+0000 7f26fd7fa640 1 -- 172.21.15.37:0/3803761710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2708036040 con 0x564735116f40 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.562+0000 7f26fd7fa640 1 -- 172.21.15.37:0/3803761710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27080042a0 con 0x564735116f40 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- 172.21.15.37:0/3803761710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 msgr2=0x5647351212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2708035000 tx=0x7f27080093e0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- 172.21.15.37:0/3803761710 shutdown_connections 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x564735123bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116f40 0x5647351212c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3803761710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735116a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- 172.21.15.37:0/3803761710 >> 172.21.15.37:0/3803761710 conn(0x564735112b30 msgr2=0x564735112f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- 172.21.15.37:0/3803761710 shutdown_connections 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- 172.21.15.37:0/3803761710 wait complete. 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- start start 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 Processor -- start 2024-04-25T21:11:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116f40 0x56473511e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x56473511e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5647351243d0 con 0x564735116620 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5647350160b0 con 0x564735121800 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564735124550 con 0x564735116f40 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.563+0000 7f271ffb0740 10 client.0 _ll_register_callbacks cb 0x564735003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47456/0 (socket says 172.21.15.37:47456) 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 -- 172.21.15.37:0/3197592772 learned_addr learned my addr 172.21.15.37:0/3197592772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116f40 0x56473511e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 -- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116f40 msgr2=0x56473511e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116f40 0x56473511e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 -- 172.21.15.37:0/3197592772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 msgr2=0x56473511e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x56473511e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 -- 172.21.15.37:0/3197592772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564734e5a620 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f27167fc640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c000 tx=0x7f270c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06a04bd9f160f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f26faff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c010840 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f270c0109e0 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2708036040 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f270c010be0 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f271ffb0740 10 client.15729 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.564+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564735130380 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2708036040 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f270c01c020 con 0x564735116620 2024-04-25T21:11:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 20 client.15729 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564735131840 0x564735133c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 4 client.15729 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564735089390 con 0x564735131840 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.565+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564735131840 0x564735133c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f26faff5640 10 client.15729 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564735131840 0x564735133c30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d96fae8302e3a7 server_cookie=910d172d20a71a56 in_seq=0 out_seq=0 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f26faff5640 1 -- 172.21.15.37:0/3197592772 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564735089390 con 0x564735131840 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f26faff5640 10 client.15729 handle_command_reply: tid=0 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f271ffb0740 1 client.15729 shutdown 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.566+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 msgr2=0x5647351210c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f270c00c000 tx=0x7f270c0039c0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 shutdown_connections 2024-04-25T21:11:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564735131840 0x564735133c30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735121800 0x56473511e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116f40 0x56473511e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3197592772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735116620 0x5647351210c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 >> 172.21.15.37:0/3197592772 conn(0x564735112b30 msgr2=0x564735118860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f26faff5640 0 client.15729 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 shutdown_connections 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 1 -- 172.21.15.37:0/3197592772 wait complete. 2024-04-25T21:11:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.567+0000 7f271ffb0740 20 client.15729 trim_cache size 0 max 16384 2024-04-25T21:11:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 Processor -- start 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 -- start start 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x5647351297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27181adab0 con 0x564735121800 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27181af5f0 con 0x564735116f40 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f271ffb0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f27181ae180 con 0x564735116620 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47472/0 (socket says 172.21.15.37:47472) 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 -- 172.21.15.37:0/3534509558 learned_addr learned my addr 172.21.15.37:0/3534509558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f2716ffd640 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x5647351297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 -- 172.21.15.37:0/3534509558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 msgr2=0x564735120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 -- 172.21.15.37:0/3534509558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 msgr2=0x5647351297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x5647351297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.582+0000 7f27167fc640 1 -- 172.21.15.37:0/3534509558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564735134290 con 0x564735121800 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f27167fc640 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f270c001eb0 tx=0x7f270c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc96fbe6c68fdb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f26fcff9640 1 -- 172.21.15.37:0/3534509558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c02b040 con 0x564735121800 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f26fcff9640 1 -- 172.21.15.37:0/3534509558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c010840 con 0x564735121800 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f26fcff9640 1 -- 172.21.15.37:0/3534509558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c01e7b0 con 0x564735121800 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 -- 172.21.15.37:0/3534509558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 msgr2=0x564735129d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f270c001eb0 tx=0x7f270c015040 comp rx=0 tx=0).stop 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 -- 172.21.15.37:0/3534509558 shutdown_connections 2024-04-25T21:11:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735129d10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x5647351297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 --2- 172.21.15.37:0/3534509558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x564735120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 -- 172.21.15.37:0/3534509558 >> 172.21.15.37:0/3534509558 conn(0x564735112b30 msgr2=0x56473511e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 -- 172.21.15.37:0/3534509558 shutdown_connections 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.583+0000 7f271ffb0740 1 -- 172.21.15.37:0/3534509558 wait complete. 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 -- start start 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 Processor -- start 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x5647351274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x564735127a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27181aef20 con 0x564735121800 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27181ae850 con 0x564735116f40 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f27181aba90 con 0x564735116620 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f271ffb0740 10 client.0 _ll_register_callbacks cb 0x7f270c00be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47474/0 (socket says 172.21.15.37:47474) 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 -- 172.21.15.37:0/2066153483 learned_addr learned my addr 172.21.15.37:0/2066153483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f2716ffd640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x564735127a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x5647351274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 -- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 msgr2=0x5647351274c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x5647351274c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 -- 172.21.15.37:0/2066153483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 msgr2=0x564735127a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x564735127a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f27167fc640 1 -- 172.21.15.37:0/2066153483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647351970a0 con 0x564735121800 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.584+0000 7f2716ffd640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x564735127a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f27167fc640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f270c00bc80 tx=0x7f270c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2555c3aeaded8589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c02b040 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f270c012bd0 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f270c010840 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f270c01e4b0 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564735198350 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c010840 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f270c031070 con 0x564735121800 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 20 client.15735 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647351988c0 0x56473519ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 4 client.15735 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.585+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564735083200 con 0x5647351988c0 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.586+0000 7f2716ffd640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647351988c0 0x56473519ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.586+0000 7f2716ffd640 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647351988c0 0x56473519ace0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f53e8015e1e907 server_cookie=30f2b416d8508928 in_seq=0 out_seq=0 2024-04-25T21:11:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.586+0000 7f26faff5640 10 client.15735 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f26faff5640 1 -- 172.21.15.37:0/2066153483 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564735083200 con 0x5647351988c0 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f26faff5640 10 client.15735 handle_command_reply: tid=0 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 client.15735 shutdown 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 msgr2=0x564735127f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f270c00bc80 tx=0x7f270c005740 comp rx=0 tx=0).stop 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 shutdown_connections 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647351988c0 0x56473519ace0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564735121800 0x564735127f60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564735116f40 0x564735127a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f26faff5640 0 client.15735 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.599+0000 7f271ffb0740 1 --2- 172.21.15.37:0/2066153483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564735116620 0x5647351274c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 >> 172.21.15.37:0/2066153483 conn(0x564735112b30 msgr2=0x56473512cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 shutdown_connections 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271ffb0740 1 -- 172.21.15.37:0/2066153483 wait complete. 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271ffb0740 20 client.15735 trim_cache size 0 max 16384 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:11:08.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 msgr2=0x7f27000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 0x7f27000a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2710098fe0 tx=0x7f2710067620 comp rx=0 tx=0).stop 2024-04-25T21:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 msgr2=0x7f271819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:08.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f271000e970 tx=0x7f271000ed70 comp rx=0 tx=0).stop 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 shutdown_connections 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271819cf80 0x7f271819d430 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2718107290 0x7f2718198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f27000a3dc0 0x7f27000a6280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 --2- 172.21.15.37:0/3017001353 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27181021c0 0x7f27181986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 >> 172.21.15.37:0/3017001353 conn(0x7f27180fda20 msgr2=0x7f27180ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 shutdown_connections 2024-04-25T21:11:08.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:08.600+0000 7f271d29e640 1 -- 172.21.15.37:0/3017001353 wait complete. 2024-04-25T21:11:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:09 smithi177 ceph-mon[33043]: pgmap v83: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:09.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:09 smithi098 ceph-mon[32569]: pgmap v83: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:09.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:09 smithi037 ceph-mon[29195]: pgmap v83: 97 pgs: 97 active+clean; 1.8 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-25T21:11:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:11 smithi177 ceph-mon[33043]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 39 op/s 2024-04-25T21:11:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:11 smithi098 ceph-mon[32569]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 39 op/s 2024-04-25T21:11:11.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:11 smithi037 ceph-mon[29195]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 57 KiB/s wr, 39 op/s 2024-04-25T21:11:13.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:13 smithi177 ceph-mon[33043]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:11:13.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:13 smithi098 ceph-mon[32569]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:11:13.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:13 smithi037 ceph-mon[29195]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:11:15.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:15 smithi177 ceph-mon[33043]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 KiB/s wr, 36 op/s 2024-04-25T21:11:15.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:15 smithi098 ceph-mon[32569]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 KiB/s wr, 36 op/s 2024-04-25T21:11:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:15 smithi037 ceph-mon[29195]: pgmap v86: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 56 KiB/s wr, 36 op/s 2024-04-25T21:11:17.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:17 smithi177 ceph-mon[33043]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:17.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:17 smithi098 ceph-mon[32569]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:17.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:17 smithi037 ceph-mon[29195]: pgmap v87: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:19 smithi177 ceph-mon[33043]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:19.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:19 smithi098 ceph-mon[32569]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:19.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:19 smithi037 ceph-mon[29195]: pgmap v88: 97 pgs: 97 active+clean; 2.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:21 smithi177 ceph-mon[33043]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:21.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:21 smithi098 ceph-mon[32569]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2872112269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 msgr2=0x7fd6d010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2872112269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d010dc70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009ca0 tx=0x7fd6c802a8b0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2872112269 shutdown_connections 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2872112269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d010dc70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2872112269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d01011a0 0x7fd6d0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2872112269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d0106fb0 0x7fd6d0107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.702+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2872112269 >> 172.21.15.37:0/2872112269 conn(0x7fd6d0076c50 msgr2=0x7fd6d0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2872112269 shutdown_connections 2024-04-25T21:11:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2872112269 wait complete. 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 Processor -- start 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 -- start start 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d01011a0 0x7fd6d0102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.703+0000 7fd6d5dad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0106fb0 0x7fd6d0103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6d5dad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0111090 con 0x7fd6d0101b60 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6d5dad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0110f10 con 0x7fd6d01011a0 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6d5dad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0111210 con 0x7fd6d0106fb0 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47594/0 (socket says 172.21.15.37:47594) 2024-04-25T21:11:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 -- 172.21.15.37:0/2927785514 learned_addr learned my addr 172.21.15.37:0/2927785514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d01011a0 0x7fd6d0102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6cffff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0106fb0 0x7fd6d0103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 -- 172.21.15.37:0/2927785514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0106fb0 msgr2=0x7fd6d0103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0106fb0 0x7fd6d0103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 -- 172.21.15.37:0/2927785514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d01011a0 msgr2=0x7fd6d0102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d01011a0 0x7fd6d0102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 -- 172.21.15.37:0/2927785514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d01a6ae0 con 0x7fd6d0101b60 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6c7fff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9f0 tx=0x7fd6b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6cd7fa640 1 -- 172.21.15.37:0/2927785514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800cd80 con 0x7fd6d0101b60 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6cd7fa640 1 -- 172.21.15.37:0/2927785514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b80054d0 con 0x7fd6d0101b60 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6cd7fa640 1 -- 172.21.15.37:0/2927785514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8010690 con 0x7fd6d0101b60 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.704+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01a6dd0 con 0x7fd6d0101b60 2024-04-25T21:11:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.705+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a7290 con 0x7fd6d0101b60 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.707+0000 7fd6cd7fa640 1 -- 172.21.15.37:0/2927785514 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6b8004e90 con 0x7fd6d0101b60 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.707+0000 7fd6d8ba0740 1 Processor -- start 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.707+0000 7fd6d8ba0740 1 -- start start 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6f40 0x5647cb3b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cd7fa640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 0x7fd6bc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5647cb30b970 con 0x5647cb3a6620 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5647cb2a84a0 con 0x5647cb3b1800 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5647cb307930 con 0x5647cb3a6f40 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47604/0 (socket says 172.21.15.37:47604) 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 -- 172.21.15.37:0/1128157428 learned_addr learned my addr 172.21.15.37:0/1128157428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6c7fff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 0x7fd6bc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cd7fa640 1 -- 172.21.15.37:0/2927785514 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd6b80d0c50 con 0x7fd6d0101b60 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6f40 0x5647cb3b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 -- 172.21.15.37:0/1128157428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6f40 msgr2=0x5647cb3b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6f40 0x5647cb3b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 -- 172.21.15.37:0/1128157428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 msgr2=0x5647cb3b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 -- 172.21.15.37:0/1128157428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cb01e210 con 0x5647cb3a6620 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6c7fff640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 0x7fd6bc0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6b809cb50 tx=0x7fd6b8092050 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.708+0000 7fd6cffff640 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8004af0 tx=0x7fd6c8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fea7bbdb71b67e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6c5ffb640 1 -- 172.21.15.37:0/1128157428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8030800 con 0x5647cb3a6620 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6c5ffb640 1 -- 172.21.15.37:0/1128157428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036040 con 0x5647cb3a6620 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6c5ffb640 1 -- 172.21.15.37:0/1128157428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004330 con 0x5647cb3a6620 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/1128157428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 msgr2=0x5647cb3a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8004af0 tx=0x7fd6c8005d80 comp rx=0 tx=0).stop 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/1128157428 shutdown_connections 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6f40 0x5647cb3b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/1128157428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6620 0x5647cb3a6a00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/1128157428 >> 172.21.15.37:0/1128157428 conn(0x5647cb3a2b30 msgr2=0x5647cb3a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/1128157428 shutdown_connections 2024-04-25T21:11:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.709+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/1128157428 wait complete. 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 -- start start 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 Processor -- start 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6620 0x5647cb3b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3b1800 0x5647cb3ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5647cb3b43d0 con 0x5647cb3a6f40 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5647cb2a60b0 con 0x5647cb3a6620 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5647cb3b4550 con 0x5647cb3b1800 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6d8ba0740 10 client.0 _ll_register_callbacks cb 0x5647cb293d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47614/0 (socket says 172.21.15.37:47614) 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.710+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/2774947218 learned_addr learned my addr 172.21.15.37:0/2774947218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cffff640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3b1800 0x5647cb3ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cffff640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6620 0x5647cb3b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3b1800 msgr2=0x5647cb3ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3b1800 0x5647cb3ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/2774947218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6620 msgr2=0x5647cb3b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6620 0x5647cb3b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/2774947218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cb0ea620 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b9f0 tx=0x7fd6c000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cc6cd0f4ca2337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0003e20 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd6c0011440 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c8036040 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6c0011640 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6d8ba0740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.711+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647cb3c03c0 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036040 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6c001b9c0 con 0x5647cb3a6f40 2024-04-25T21:11:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 10 client.15747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 20 client.15747 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb3c18d0 0x5647cb3c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 4 client.15747 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647cb319390 con 0x5647cb3c18d0 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.712+0000 7fd6cffff640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb3c18d0 0x5647cb3c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.713+0000 7fd6cffff640 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb3c18d0 0x5647cb3c3d20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a1654c6b80db49 server_cookie=b0df0c7b43df90d1 in_seq=0 out_seq=0 2024-04-25T21:11:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.713+0000 7fd6aeffd640 10 client.15747 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.713+0000 7fd6aeffd640 1 -- 172.21.15.37:0/2774947218 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647cb319390 con 0x5647cb3c18d0 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.713+0000 7fd6aeffd640 10 client.15747 handle_command_reply: tid=0 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 client.15747 shutdown 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 msgr2=0x5647cb3ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b9f0 tx=0x7fd6c000beb0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 shutdown_connections 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb3c18d0 0x5647cb3c3d20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3b1800 0x5647cb3ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3ae3d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6aeffd640 0 client.15747 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/2774947218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6620 0x5647cb3b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 >> 172.21.15.37:0/2774947218 conn(0x5647cb3a2b30 msgr2=0x5647cb3a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 shutdown_connections 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/2774947218 wait complete. 2024-04-25T21:11:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.714+0000 7fd6d8ba0740 20 client.15747 trim_cache size 0 max 16384 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 Processor -- start 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 -- start start 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6f40 0x5647cb3a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01afbe0 con 0x5647cb3b1800 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01b1720 con 0x5647cb3a6f40 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01b02b0 con 0x5647cb3a6620 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.729+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47626/0 (socket says 172.21.15.37:47626) 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 -- 172.21.15.37:0/3048229699 learned_addr learned my addr 172.21.15.37:0/3048229699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6f40 0x5647cb3a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 -- 172.21.15.37:0/3048229699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 msgr2=0x5647cb3bd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3bd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 -- 172.21.15.37:0/3048229699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6f40 msgr2=0x5647cb3a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6f40 0x5647cb3a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 -- 172.21.15.37:0/3048229699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cb3a9eb0 con 0x5647cb3b1800 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009960 tx=0x7fd6c803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62d15f6db48d885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6c57fa640 1 -- 172.21.15.37:0/3048229699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c805a070 con 0x5647cb3b1800 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6c57fa640 1 -- 172.21.15.37:0/3048229699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036070 con 0x5647cb3b1800 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.730+0000 7fd6c57fa640 1 -- 172.21.15.37:0/3048229699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80316d0 con 0x5647cb3b1800 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3048229699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 msgr2=0x5647cb3b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009960 tx=0x7fd6c803d040 comp rx=0 tx=0).stop 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3048229699 shutdown_connections 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3b1800 0x5647cb3b0c10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3a6f40 0x5647cb3a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3048229699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3bd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3048229699 >> 172.21.15.37:0/3048229699 conn(0x5647cb3a2b30 msgr2=0x5647cb3a9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3048229699 shutdown_connections 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3048229699 wait complete. 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 -- start start 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 Processor -- start 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.731+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01b1050 con 0x5647cb3a6f40 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01b0980 con 0x5647cb3b1800 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d01adbc0 con 0x5647cb3a6620 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 10 client.0 _ll_register_callbacks cb 0x7fd6c00044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47632/0 (socket says 172.21.15.37:47632) 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/3324577213 learned_addr learned my addr 172.21.15.37:0/3324577213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 msgr2=0x5647cb3b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/3324577213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 msgr2=0x5647cb3ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 -- 172.21.15.37:0/3324577213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647cb3ab960 con 0x5647cb3a6f40 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6cf7fe640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001eb0 tx=0x7fd6c0004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa07f9613b483d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0028040 con 0x5647cb3a6f40 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fd6c0005180 con 0x5647cb3a6f40 2024-04-25T21:11:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6c8036070 con 0x5647cb3a6f40 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6c0005380 con 0x5647cb3a6f40 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 10 client.15753 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.732+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5647cb427de0 con 0x5647cb3a6f40 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c8036070 con 0x5647cb3a6f40 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6c002b070 con 0x5647cb3a6f40 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 10 client.15753 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 20 client.15753 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb428580 0x5647cb42a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 4 client.15753 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647cb313200 con 0x5647cb428580 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.733+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb428580 0x5647cb42a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.734+0000 7fd6cffff640 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb428580 0x5647cb42a9a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a545f4529242531 server_cookie=e40d92ecde224dc in_seq=0 out_seq=0 2024-04-25T21:11:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.734+0000 7fd6aeffd640 10 client.15753 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:21.740 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:21 smithi037 ceph-mon[29195]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6aeffd640 1 -- 172.21.15.37:0/3324577213 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647cb313200 con 0x5647cb428580 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6aeffd640 10 client.15753 handle_command_reply: tid=0 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 client.15753 shutdown 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 msgr2=0x5647cb3aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001eb0 tx=0x7fd6c0004050 comp rx=0 tx=0).stop 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 shutdown_connections 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5647cb428580 0x5647cb42a9a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5647cb3b1800 0x5647cb3ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647cb3a6f40 0x5647cb3aace0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6aeffd640 0 client.15753 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 --2- 172.21.15.37:0/3324577213 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5647cb3a6620 0x5647cb3b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 >> 172.21.15.37:0/3324577213 conn(0x5647cb3a2b30 msgr2=0x5647cb3a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 shutdown_connections 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 1 -- 172.21.15.37:0/3324577213 wait complete. 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.747+0000 7fd6d8ba0740 20 client.15753 trim_cache size 0 max 16384 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:11:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 msgr2=0x7fd6bc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 0x7fd6bc0a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6b809cb50 tx=0x7fd6b8092050 comp rx=0 tx=0).stop 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 msgr2=0x7fd6d0103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9f0 tx=0x7fd6b800edf0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 shutdown_connections 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6bc0a3e40 0x7fd6bc0a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6d0106fb0 0x7fd6d0103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d0101b60 0x7fd6d0103050 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 --2- 172.21.15.37:0/2927785514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d01011a0 0x7fd6d0102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 >> 172.21.15.37:0/2927785514 conn(0x7fd6d0076c50 msgr2=0x7fd6d01003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 shutdown_connections 2024-04-25T21:11:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:21.748+0000 7fd6d5dad640 1 -- 172.21.15.37:0/2927785514 wait complete. 2024-04-25T21:11:23.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:23 smithi177 ceph-mon[33043]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T21:11:23.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:23 smithi098 ceph-mon[32569]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T21:11:23.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:23 smithi037 ceph-mon[29195]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-25T21:11:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:25 smithi177 ceph-mon[33043]: pgmap v91: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-04-25T21:11:25.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:25 smithi098 ceph-mon[32569]: pgmap v91: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-04-25T21:11:25.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:25 smithi037 ceph-mon[29195]: pgmap v91: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 21 op/s 2024-04-25T21:11:27.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:27 smithi098 ceph-mon[32569]: pgmap v92: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:27.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:27 smithi037 ceph-mon[29195]: pgmap v92: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:27 smithi177 ceph-mon[33043]: pgmap v92: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:29.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:29 smithi098 ceph-mon[32569]: pgmap v93: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:29.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:29 smithi037 ceph-mon[29195]: pgmap v93: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:29.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:29 smithi177 ceph-mon[33043]: pgmap v93: 97 pgs: 97 active+clean; 2.3 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:31.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:31 smithi098 ceph-mon[32569]: pgmap v94: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:31.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:31 smithi037 ceph-mon[29195]: pgmap v94: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:31 smithi177 ceph-mon[33043]: pgmap v94: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:33.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:33 smithi098 ceph-mon[32569]: pgmap v95: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:33.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:33 smithi037 ceph-mon[29195]: pgmap v95: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:33 smithi177 ceph-mon[33043]: pgmap v95: 97 pgs: 97 active+clean; 2.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 -- 172.21.15.37:0/2524084663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103b20 msgr2=0x7fa2d810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 --2- 172.21.15.37:0/2524084663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103b20 0x7fa2d810c000 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009900 tx=0x7fa2cc02a480 comp rx=0 tx=0).stop 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 -- 172.21.15.37:0/2524084663 shutdown_connections 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 --2- 172.21.15.37:0/2524084663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d810c5d0 0x7fa2d810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 --2- 172.21.15.37:0/2524084663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d8103b20 0x7fa2d810c000 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 --2- 172.21.15.37:0/2524084663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 0x7fa2d81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 -- 172.21.15.37:0/2524084663 >> 172.21.15.37:0/2524084663 conn(0x7fa2d80771d0 msgr2=0x7fa2d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 -- 172.21.15.37:0/2524084663 shutdown_connections 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.849+0000 7fa2debf7640 1 -- 172.21.15.37:0/2524084663 wait complete. 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 Processor -- start 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 -- start start 2024-04-25T21:11:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 0x7fa2d819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 0x7fa2d819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112210 con 0x7fa2d810c5d0 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112090 con 0x7fa2d8103b20 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2debf7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112390 con 0x7fa2d8103200 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2dd194640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2dd194640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53104/0 (socket says 172.21.15.37:53104) 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2dd194640 1 -- 172.21.15.37:0/1231074575 learned_addr learned my addr 172.21.15.37:0/1231074575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2dc993640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 0x7fa2d819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.850+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 0x7fa2d819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 -- 172.21.15.37:0/1231074575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 msgr2=0x7fa2d819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 0x7fa2d819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 -- 172.21.15.37:0/1231074575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 msgr2=0x7fa2d819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 0x7fa2d819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 -- 172.21.15.37:0/1231074575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d8198d90 con 0x7fa2d810c5d0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 0x7fa2d819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2dd194640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000dac0 tx=0x7fa2d000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2d5ffb640 1 -- 172.21.15.37:0/1231074575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0017070 con 0x7fa2d810c5d0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2d5ffb640 1 -- 172.21.15.37:0/1231074575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d000be30 con 0x7fa2d810c5d0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2d5ffb640 1 -- 172.21.15.37:0/1231074575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0010460 con 0x7fa2d810c5d0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d8199080 con 0x7fa2d810c5d0 2024-04-25T21:11:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.851+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a52d0 con 0x7fa2d810c5d0 2024-04-25T21:11:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.853+0000 7fa2d5ffb640 1 -- 172.21.15.37:0/1231074575 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa2d00040b0 con 0x7fa2d810c5d0 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 Processor -- start 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 -- start start 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d5ffb640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 0x7fa2c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf210ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dcf2070970 con 0x55dcf210bf40 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dcf200d4a0 con 0x55dcf210b620 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dcf206c930 con 0x55dcf2116800 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 0x7fa2c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d5ffb640 1 -- 172.21.15.37:0/1231074575 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa2d00d00d0 con 0x7fa2d810c5d0 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53106/0 (socket says 172.21.15.37:53106) 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.854+0000 7fa2d7fff640 1 -- 172.21.15.37:0/209278695 learned_addr learned my addr 172.21.15.37:0/209278695 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2dc993640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf210ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2dd194640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 -- 172.21.15.37:0/209278695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 msgr2=0x55dcf2118bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2118bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 -- 172.21.15.37:0/209278695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 msgr2=0x55dcf210ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf210ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 -- 172.21.15.37:0/209278695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf1d83210 con 0x55dcf210bf40 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2dc993640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf210ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 0x7fa2c00a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc032fd0 tx=0x7fa2cc032ba0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005940 tx=0x7fa2cc045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635baffda39145f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2beffd640 1 -- 172.21.15.37:0/209278695 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc035040 con 0x55dcf210bf40 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2beffd640 1 -- 172.21.15.37:0/209278695 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc003e70 con 0x55dcf210bf40 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.855+0000 7fa2beffd640 1 -- 172.21.15.37:0/209278695 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc03da80 con 0x55dcf210bf40 2024-04-25T21:11:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- 172.21.15.37:0/209278695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 msgr2=0x55dcf21162c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005940 tx=0x7fa2cc045000 comp rx=0 tx=0).stop 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- 172.21.15.37:0/209278695 shutdown_connections 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf21162c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/209278695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf210ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- 172.21.15.37:0/209278695 >> 172.21.15.37:0/209278695 conn(0x55dcf2107b30 msgr2=0x55dcf2107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- 172.21.15.37:0/209278695 shutdown_connections 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- 172.21.15.37:0/209278695 wait complete. 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 -- start start 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.856+0000 7fa2e19e5740 1 Processor -- start 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2116130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2113960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dcf21193d0 con 0x55dcf210bf40 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55dcf200b0b0 con 0x55dcf210b620 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55dcf2119550 con 0x55dcf2116800 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2e19e5740 10 client.0 _ll_register_callbacks cb 0x55dcf1ff8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53108/0 (socket says 172.21.15.37:53108) 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2690411421 learned_addr learned my addr 172.21.15.37:0/2690411421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2dd194640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 msgr2=0x55dcf2113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2690411421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 msgr2=0x55dcf2116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2116130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2690411421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf1e4f620 con 0x55dcf210bf40 2024-04-25T21:11:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.857+0000 7fa2dd194640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc000c00 tx=0x7fa2cc004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee327e69b14c4d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc056070 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa2cc0039c0 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc003e70 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc035040 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2e19e5740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcf2125320 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc003e70 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2cc03a040 con 0x55dcf210bf40 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.858+0000 7fa2e19e5740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2e19e5740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2e19e5740 20 client.15765 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf2126670 0x55dcf2128a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2e19e5740 4 client.15765 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:11:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcf207e390 con 0x55dcf2126670 2024-04-25T21:11:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf2126670 0x55dcf2128a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2b7fff640 10 client.15765 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.859+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf2126670 0x55dcf2128a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7caa6f67c048342f server_cookie=829d9a18e76b661a in_seq=0 out_seq=0 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2690411421 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcf207e390 con 0x55dcf2126670 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2b7fff640 10 client.15765 handle_command_reply: tid=0 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 client.15765 shutdown 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 msgr2=0x55dcf2113420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc000c00 tx=0x7fa2cc004020 comp rx=0 tx=0).stop 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 shutdown_connections 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf2126670 0x55dcf2128a60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2113420 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2690411421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2116130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2b7fff640 0 client.15765 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 >> 172.21.15.37:0/2690411421 conn(0x55dcf2107b30 msgr2=0x55dcf210d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 shutdown_connections 2024-04-25T21:11:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.860+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2690411421 wait complete. 2024-04-25T21:11:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.861+0000 7fa2e19e5740 20 client.15765 trim_cache size 0 max 16384 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 Processor -- start 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 -- start start 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2122290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf212e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ada50 con 0x55dcf210bf40 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81af590 con 0x55dcf210b620 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ae120 con 0x55dcf2116800 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53112/0 (socket says 172.21.15.37:53112) 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2867075829 learned_addr learned my addr 172.21.15.37:0/2867075829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2122290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2dd194640 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf212e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2867075829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 msgr2=0x55dcf212e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf212e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2867075829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 msgr2=0x55dcf2122290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2122290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.876+0000 7fa2d7fff640 1 -- 172.21.15.37:0/2867075829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcf2128dd0 con 0x55dcf210bf40 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc05b620 tx=0x7fa2cc031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18a7eff1107e59a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2be7fc640 1 -- 172.21.15.37:0/2867075829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc069030 con 0x55dcf210bf40 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2be7fc640 1 -- 172.21.15.37:0/2867075829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc003e70 con 0x55dcf210bf40 2024-04-25T21:11:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2be7fc640 1 -- 172.21.15.37:0/2867075829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc056070 con 0x55dcf210bf40 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2867075829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 msgr2=0x55dcf2115c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.877+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc05b620 tx=0x7fa2cc031520 comp rx=0 tx=0).stop 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2867075829 shutdown_connections 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf212e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210bf40 0x55dcf2115c60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2867075829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210b620 0x55dcf2122290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2867075829 >> 172.21.15.37:0/2867075829 conn(0x55dcf2107b30 msgr2=0x55dcf2106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2867075829 shutdown_connections 2024-04-25T21:11:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2867075829 wait complete. 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- start start 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 Processor -- start 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 0x55dcf210fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2110420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81aeec0 con 0x55dcf210b620 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ae7f0 con 0x55dcf210bf40 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.878+0000 7fa2e19e5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81aba30 con 0x55dcf2116800 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2e19e5740 10 client.0 _ll_register_callbacks cb 0x7fa2cc0041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53116/0 (socket says 172.21.15.37:53116) 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 -- 172.21.15.37:0/2089681492 learned_addr learned my addr 172.21.15.37:0/2089681492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 0x55dcf210fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dd194640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2110420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 -- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 msgr2=0x55dcf2110420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2110420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 -- 172.21.15.37:0/2089681492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 msgr2=0x55dcf210fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 0x55dcf210fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 -- 172.21.15.37:0/2089681492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcf2110c50 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 0x55dcf210fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2dc993640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005d00 tx=0x7fa2c801f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2352b56708bccc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005220 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fa2c80053c0 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc003e70 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.879+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2c802e050 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcf218c950 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc003e70 con 0x55dcf210b620 2024-04-25T21:11:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa2c80039c0 con 0x55dcf210b620 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 20 client.15771 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf218d100 0x55dcf218f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 4 client.15771 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcf2078200 con 0x55dcf218d100 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.880+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf218d100 0x55dcf218f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.881+0000 7fa2b7fff640 10 client.15771 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.881+0000 7fa2d7fff640 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf218d100 0x55dcf218f520 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f05aee840506b7 server_cookie=67d8130216313757 in_seq=0 out_seq=0 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2b7fff640 1 -- 172.21.15.37:0/2089681492 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcf2078200 con 0x55dcf218d100 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2b7fff640 10 client.15771 handle_command_reply: tid=0 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 client.15771 shutdown 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 msgr2=0x55dcf211ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005d00 tx=0x7fa2c801f7f0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 shutdown_connections 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55dcf218d100 0x55dcf218f520 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dcf2116800 0x55dcf2110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55dcf210bf40 0x55dcf210fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 --2- 172.21.15.37:0/2089681492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dcf210b620 0x55dcf211ec80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2b7fff640 0 client.15771 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 >> 172.21.15.37:0/2089681492 conn(0x55dcf2107b30 msgr2=0x55dcf2111be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 shutdown_connections 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 1 -- 172.21.15.37:0/2089681492 wait complete. 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2e19e5740 20 client.15771 trim_cache size 0 max 16384 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:11:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:11:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 msgr2=0x7fa2c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 0x7fa2c00a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc032fd0 tx=0x7fa2cc032ba0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 msgr2=0x7fa2d8198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000dac0 tx=0x7fa2d000df80 comp rx=0 tx=0).stop 2024-04-25T21:11:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 shutdown_connections 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2d810c5d0 0x7fa2d8198610 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d8103b20 0x7fa2d819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa2c00a3dc0 0x7fa2c00a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 --2- 172.21.15.37:0/1231074575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2d8103200 0x7fa2d819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 >> 172.21.15.37:0/1231074575 conn(0x7fa2d80771d0 msgr2=0x7fa2d80ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 shutdown_connections 2024-04-25T21:11:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:34.896+0000 7fa2debf7640 1 -- 172.21.15.37:0/1231074575 wait complete. 2024-04-25T21:11:35.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:35 smithi098 ceph-mon[32569]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:35.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:35 smithi037 ceph-mon[29195]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:35.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:35 smithi177 ceph-mon[33043]: pgmap v96: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:36.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:36 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:11:36.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:36 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:11:36.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:36 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:11:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:37 smithi098 ceph-mon[32569]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:37 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:11:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:37 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:11:37.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:37 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:11:37.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:37 smithi037 ceph-mon[29195]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:37.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:37 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:11:37.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:37 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:11:37.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:37 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:11:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:37 smithi177 ceph-mon[33043]: pgmap v97: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:37 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:11:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:37 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:11:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:37 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:11:39.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:39 smithi098 ceph-mon[32569]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:39 smithi037 ceph-mon[29195]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:39 smithi177 ceph-mon[33043]: pgmap v98: 97 pgs: 97 active+clean; 2.5 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:11:41.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:41 smithi098 ceph-mon[32569]: pgmap v99: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:41.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:41 smithi037 ceph-mon[29195]: pgmap v99: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:41 smithi177 ceph-mon[33043]: pgmap v99: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:1/600: open fd = 3 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:2/600: open fd = 3 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.002 INFO:tasks.workunit.client.0.smithi037.stdout:3/600: open fd = 3 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:4/600: open fd = 3 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:5/600: open fd = 3 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:6/600: open fd = 3 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.003 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:7/600: open fd = 3 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:8/600: open fd = 3 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:9/600: open fd = 3 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:10/600: open fd = 3 2024-04-25T21:11:43.004 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:11/600: open fd = 3 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:12/600: open fd = 3 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:13/600: open fd = 3 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.005 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:14/600: open fd = 3 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:15/600: open fd = 3 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:16/600: open fd = 3 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.006 INFO:tasks.workunit.client.0.smithi037.stdout:17/600: open fd = 3 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:18/600: open fd = 3 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:19/600: open fd = 3 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:20/600: open fd = 3 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.007 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:21/600: open fd = 3 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:22/600: open fd = 3 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:23/600: open fd = 3 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:24/600: open fd = 3 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.008 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:25/600: open fd = 3 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:26/600: open fd = 3 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:27/600: open fd = 3 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:28/600: open fd = 3 2024-04-25T21:11:43.009 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:29/600: open fd = 3 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:30/600: open fd = 3 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:31/600: open fd = 3 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.010 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:32/600: open fd = 3 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:33/600: open fd = 3 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:34/600: open fd = 3 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:35/600: open fd = 3 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.011 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:36/600: open fd = 3 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:37/600: open fd = 3 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:38/600: open fd = 3 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:39/600: open fd = 3 2024-04-25T21:11:43.012 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:40/600: open fd = 3 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:41/600: open fd = 3 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:42/600: open fd = 3 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.013 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:43/600: open fd = 3 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:44/600: open fd = 3 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:45/600: open fd = 3 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:46/600: open fd = 3 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.014 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:47/600: open fd = 3 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:48/600: open fd = 3 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:49/600: open fd = 3 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:50/600: open fd = 3 2024-04-25T21:11:43.015 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:51/600: open fd = 3 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:52/600: open fd = 3 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:53/600: open fd = 3 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.016 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:54/600: open fd = 3 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:55/600: open fd = 3 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:56/600: open fd = 3 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:57/600: open fd = 3 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.017 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:58/600: open fd = 3 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:59/600: open fd = 3 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:60/600: open fd = 3 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.018 INFO:tasks.workunit.client.0.smithi037.stdout:61/600: open fd = 3 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:62/600: open fd = 3 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:63/600: open fd = 3 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:11:43.019 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:11:43.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:43 smithi098 ceph-mon[32569]: pgmap v100: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T21:11:43.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:43 smithi037 ceph-mon[29195]: pgmap v100: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T21:11:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:43 smithi177 ceph-mon[33043]: pgmap v100: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T21:11:45.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:45 smithi098 ceph-mon[32569]: pgmap v101: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:45.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:45 smithi037 ceph-mon[29195]: pgmap v101: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:45 smithi177 ceph-mon[33043]: pgmap v101: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-25T21:11:47.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:47 smithi098 ceph-mon[32569]: pgmap v102: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:47.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:47 smithi037 ceph-mon[29195]: pgmap v102: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:47 smithi177 ceph-mon[33043]: pgmap v102: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.996+0000 7f66a0576640 1 -- 172.21.15.37:0/3379541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698107290 msgr2=0x7f669810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.996+0000 7f66a0576640 1 --2- 172.21.15.37:0/3379541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698107290 0x7f669810ee60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6690009ca0 tx=0x7f669002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 -- 172.21.15.37:0/3379541735 shutdown_connections 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 --2- 172.21.15.37:0/3379541735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6698107290 0x7f669810ee60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 --2- 172.21.15.37:0/3379541735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6698102b70 0x7f6698106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 --2- 172.21.15.37:0/3379541735 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66981021c0 0x7f66981025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 -- 172.21.15.37:0/3379541735 >> 172.21.15.37:0/3379541735 conn(0x7f66980fda40 msgr2=0x7f66980ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 -- 172.21.15.37:0/3379541735 shutdown_connections 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 -- 172.21.15.37:0/3379541735 wait complete. 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 Processor -- start 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.997+0000 7f66a0576640 1 -- start start 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6698107290 0x7f6698198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f669819cee0 0x7f669819d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112200 con 0x7f66981021c0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112080 con 0x7f6698107290 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f66a0576640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112380 con 0x7f669819cee0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54358/0 (socket says 172.21.15.37:54358) 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 -- 172.21.15.37:0/455603195 learned_addr learned my addr 172.21.15.37:0/455603195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669db11640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6698107290 0x7f6698198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669eb13640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f669819cee0 0x7f669819d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 -- 172.21.15.37:0/455603195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f669819cee0 msgr2=0x7f669819d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f669819cee0 0x7f669819d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 -- 172.21.15.37:0/455603195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6698107290 msgr2=0x7f6698198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6698107290 0x7f6698198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 -- 172.21.15.37:0/455603195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819d9d0 con 0x7f66981021c0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f669e312640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f669400c920 tx=0x7f669400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.998+0000 7f668f7fe640 1 -- 172.21.15.37:0/455603195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694007c70 con 0x7f66981021c0 2024-04-25T21:11:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.999+0000 7f668f7fe640 1 -- 172.21.15.37:0/455603195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694007e10 con 0x7f66981021c0 2024-04-25T21:11:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.999+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819dcc0 con 0x7f66981021c0 2024-04-25T21:11:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.999+0000 7f668f7fe640 1 -- 172.21.15.37:0/455603195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018820 con 0x7f66981021c0 2024-04-25T21:11:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:47.999+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a54b0 con 0x7f66981021c0 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.001+0000 7f668f7fe640 1 -- 172.21.15.37:0/455603195 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f669400f450 con 0x7f66981021c0 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.001+0000 7f66a336f740 1 Processor -- start 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 -- start start 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 0x564b06cc58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cd4c10 0x564b06cd7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b06c2a970 con 0x564b06cc5e20 2024-04-25T21:11:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564b06bc74a0 con 0x564b06cc5500 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f66a336f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564b06c26930 con 0x564b06cd4c10 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54368/0 (socket says 172.21.15.37:54368) 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669e312640 1 -- 172.21.15.37:0/1698162456 learned_addr learned my addr 172.21.15.37:0/1698162456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669eb13640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 0x564b06cc58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f668f7fe640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 0x7f66840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669db11640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cd4c10 0x564b06cd7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.002+0000 7f669eb13640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 0x7f66840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 -- 172.21.15.37:0/1698162456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cd4c10 msgr2=0x564b06cd7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cd4c10 0x564b06cd7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f668f7fe640 1 -- 172.21.15.37:0/455603195 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f66940d08f0 con 0x7f66981021c0 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 -- 172.21.15.37:0/1698162456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 msgr2=0x564b06cd46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 -- 172.21.15.37:0/1698162456 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0693d210 con 0x564b06cc5500 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669e312640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 0x564b06cc58e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f6690031450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd861500659d867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f669eb13640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 0x7f66840a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6690047000 tx=0x7f6690049000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f668cbfe640 1 -- 172.21.15.37:0/1698162456 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690031af0 con 0x564b06cc5500 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f668cbfe640 1 -- 172.21.15.37:0/1698162456 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6690036040 con 0x564b06cc5500 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.003+0000 7f668cbfe640 1 -- 172.21.15.37:0/1698162456 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669002fdb0 con 0x564b06cc5500 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- 172.21.15.37:0/1698162456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 msgr2=0x564b06cc58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 0x564b06cc58e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f6690031450 comp rx=0 tx=0).stop 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- 172.21.15.37:0/1698162456 shutdown_connections 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cd4c10 0x564b06cd7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 --2- 172.21.15.37:0/1698162456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cc5500 0x564b06cc58e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- 172.21.15.37:0/1698162456 >> 172.21.15.37:0/1698162456 conn(0x564b06cc1b30 msgr2=0x564b06cc1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- 172.21.15.37:0/1698162456 shutdown_connections 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- 172.21.15.37:0/1698162456 wait complete. 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 -- start start 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.004+0000 7f66a336f740 1 Processor -- start 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5500 0x564b06cd4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564b06cd7780 con 0x564b06cc5e20 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564b06bc50b0 con 0x564b06cd4c10 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564b06cd7900 con 0x564b06cc5500 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f66a336f740 10 client.0 _ll_register_callbacks cb 0x564b06bb2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54380/0 (socket says 172.21.15.37:54380) 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 -- 172.21.15.37:0/368845360 learned_addr learned my addr 172.21.15.37:0/368845360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669eb13640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669e312640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5500 0x564b06cd4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 -- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5500 msgr2=0x564b06cd4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5500 0x564b06cd4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 -- 172.21.15.37:0/368845360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 msgr2=0x564b06cd1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 -- 172.21.15.37:0/368845360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b06a09620 con 0x564b06cc5e20 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669eb13640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f669db11640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f668800b920 tx=0x7f668800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70da23ae42ebfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f6681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.005+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688005020 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6690036040 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f66880051c0 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66880053c0 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b06ce3800 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6690036040 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f668801b500 con 0x564b06cc5e20 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 20 client.15783 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06ce4cc0 0x564b06ce70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 4 client.15783 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:11:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.006+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b06c38390 con 0x564b06ce4cc0 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.007+0000 7f669e312640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06ce4cc0 0x564b06ce70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.007+0000 7f669e312640 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06ce4cc0 0x564b06ce70b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7baaf53fbaec6b99 server_cookie=c99db6337bd664af in_seq=0 out_seq=0 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.007+0000 7f6681ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f6681ffb640 1 -- 172.21.15.37:0/368845360 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b06c38390 con 0x564b06ce4cc0 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f6681ffb640 10 client.15783 handle_command_reply: tid=0 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 client.15783 shutdown 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 msgr2=0x564b06cd1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f668800b920 tx=0x7f668800bde0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 shutdown_connections 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06ce4cc0 0x564b06ce70b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5e20 0x564b06cd1880 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f6681ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 --2- 172.21.15.37:0/368845360 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5500 0x564b06cd4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 >> 172.21.15.37:0/368845360 conn(0x564b06cc1b30 msgr2=0x564b06cc78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 shutdown_connections 2024-04-25T21:11:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 1 -- 172.21.15.37:0/368845360 wait complete. 2024-04-25T21:11:48.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.008+0000 7f66a336f740 20 client.15783 trim_cache size 0 max 16384 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 Processor -- start 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 -- start start 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cc0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ab7c0 con 0x564b06cc5500 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ad300 con 0x564b06cd4c10 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.023+0000 7f66a336f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66981abe90 con 0x564b06cc5e20 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54384/0 (socket says 172.21.15.37:54384) 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 -- 172.21.15.37:0/2802286654 learned_addr learned my addr 172.21.15.37:0/2802286654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669db11640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cc0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 -- 172.21.15.37:0/2802286654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 msgr2=0x564b06cc0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cc0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669eb13640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 -- 172.21.15.37:0/2802286654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 msgr2=0x564b06cd40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 -- 172.21.15.37:0/2802286654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b06cd2170 con 0x564b06cc5500 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669eb13640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f669e312640 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6694007590 tx=0x7f66940bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930d41f9ca9a62a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f6683fff640 1 -- 172.21.15.37:0/2802286654 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66940a5530 con 0x564b06cc5500 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f6683fff640 1 -- 172.21.15.37:0/2802286654 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669409c8c0 con 0x564b06cc5500 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.024+0000 7f6683fff640 1 -- 172.21.15.37:0/2802286654 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669400f740 con 0x564b06cc5500 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- 172.21.15.37:0/2802286654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 msgr2=0x564b06ce0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6694007590 tx=0x7f66940bd040 comp rx=0 tx=0).stop 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- 172.21.15.37:0/2802286654 shutdown_connections 2024-04-25T21:11:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cd40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cc0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 --2- 172.21.15.37:0/2802286654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06ce0680 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- 172.21.15.37:0/2802286654 >> 172.21.15.37:0/2802286654 conn(0x564b06cc1b30 msgr2=0x564b06cc0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- 172.21.15.37:0/2802286654 shutdown_connections 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- 172.21.15.37:0/2802286654 wait complete. 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 -- start start 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.025+0000 7f66a336f740 1 Processor -- start 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f66981acc30 con 0x564b06cc5500 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f66981ac560 con 0x564b06cd4c10 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f66981a97a0 con 0x564b06cc5e20 2024-04-25T21:11:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 10 client.0 _ll_register_callbacks cb 0x7f6688004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54396/0 (socket says 172.21.15.37:54396) 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 -- 172.21.15.37:0/1743103105 learned_addr learned my addr 172.21.15.37:0/1743103105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669eb13640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669db11640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 -- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 msgr2=0x564b06cce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 -- 172.21.15.37:0/1743103105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 msgr2=0x564b06cce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 -- 172.21.15.37:0/1743103105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b06ccf040 con 0x564b06cc5500 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f669e312640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f66940a5000 tx=0x7f66940a0d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b1336c948ccb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f6681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669400f740 con 0x564b06cc5500 2024-04-25T21:11:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6694098f30 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669409c8c0 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66940ce2f0 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.026+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564b06d4b1d0 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669409c8c0 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66940a50a0 con 0x564b06cc5500 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 20 client.15789 populate_metadata read hostname 'smithi037' 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06d4b890 0x564b06d4dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 4 client.15789 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b06c32200 con 0x564b06d4b890 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.027+0000 7f669db11640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06d4b890 0x564b06d4dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.028+0000 7f6681ffb640 10 client.15789 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.028+0000 7f669db11640 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06d4b890 0x564b06d4dc80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce5b0006375477e4 server_cookie=c3bbbc58a5a91458 in_seq=0 out_seq=0 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f6681ffb640 1 -- 172.21.15.37:0/1743103105 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b06c32200 con 0x564b06d4b890 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f6681ffb640 10 client.15789 handle_command_reply: tid=0 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 client.15789 shutdown 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 msgr2=0x564b06cdcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f66940a5000 tx=0x7f66940a0d60 comp rx=0 tx=0).stop 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 shutdown_connections 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564b06d4b890 0x564b06d4dc80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564b06cd4c10 0x564b06cce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564b06cc5e20 0x564b06cce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f6681ffb640 0 client.15789 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:11:48.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 --2- 172.21.15.37:0/1743103105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564b06cc5500 0x564b06cdcf40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 >> 172.21.15.37:0/1743103105 conn(0x564b06cc1b30 msgr2=0x564b06cc0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 shutdown_connections 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 1 -- 172.21.15.37:0/1743103105 wait complete. 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.044+0000 7f66a336f740 20 client.15789 trim_cache size 0 max 16384 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:11:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:11:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 msgr2=0x7f66840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 0x7f66840a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6690047000 tx=0x7f6690049000 comp rx=0 tx=0).stop 2024-04-25T21:11:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 msgr2=0x7f6698198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:11:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f669400c920 tx=0x7f669400cde0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 shutdown_connections 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66840a3dc0 0x7f66840a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f669819cee0 0x7f669819d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6698107290 0x7f6698198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 --2- 172.21.15.37:0/455603195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66981021c0 0x7f6698198620 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 >> 172.21.15.37:0/455603195 conn(0x7f66980fda40 msgr2=0x7f66980fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 shutdown_connections 2024-04-25T21:11:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:11:48.045+0000 7f66a0576640 1 -- 172.21.15.37:0/455603195 wait complete. 2024-04-25T21:11:49.642 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:49 smithi177 ceph-mon[33043]: pgmap v103: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:49.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:49 smithi098 ceph-mon[32569]: pgmap v103: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:49.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:49 smithi037 ceph-mon[29195]: pgmap v103: 97 pgs: 97 active+clean; 2.8 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:51.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:51 smithi098 ceph-mon[32569]: pgmap v104: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-04-25T21:11:51.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:51 smithi037 ceph-mon[29195]: pgmap v104: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-04-25T21:11:51.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:51 smithi177 ceph-mon[33043]: pgmap v104: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-04-25T21:11:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:53 smithi098 ceph-mon[32569]: pgmap v105: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:53.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:53 smithi037 ceph-mon[29195]: pgmap v105: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:53.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:53 smithi177 ceph-mon[33043]: pgmap v105: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-25T21:11:55.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:55 smithi098 ceph-mon[32569]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:55.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:55 smithi037 ceph-mon[29195]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:55 smithi177 ceph-mon[33043]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:11:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:57 smithi098 ceph-mon[32569]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:11:57.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:57 smithi037 ceph-mon[29195]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:11:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:57 smithi177 ceph-mon[33043]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:11:59.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:11:59 smithi098 ceph-mon[32569]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:11:59.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:11:59 smithi037 ceph-mon[29195]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:11:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:11:59 smithi177 ceph-mon[33043]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-25T21:12:01.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 -- 172.21.15.37:0/2111943666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 msgr2=0x7feec0107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 --2- 172.21.15.37:0/2111943666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0107390 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc02a400 comp rx=0 tx=0).stop 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 -- 172.21.15.37:0/2111943666 shutdown_connections 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 --2- 172.21.15.37:0/2111943666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec0101b60 0x7feec010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 --2- 172.21.15.37:0/2111943666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec01011a0 0x7feec0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 --2- 172.21.15.37:0/2111943666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0107390 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 -- 172.21.15.37:0/2111943666 >> 172.21.15.37:0/2111943666 conn(0x7feec0076c50 msgr2=0x7feec0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 -- 172.21.15.37:0/2111943666 shutdown_connections 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 -- 172.21.15.37:0/2111943666 wait complete. 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.144+0000 7feec777d640 1 Processor -- start 2024-04-25T21:12:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 -- start start 2024-04-25T21:12:01.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec01011a0 0x7feec0102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec0101b60 0x7feec0103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feec0111090 con 0x7feec0106fb0 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feec0110f10 con 0x7feec01011a0 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec777d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feec0111210 con 0x7feec0101b60 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40196/0 (socket says 172.21.15.37:40196) 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 -- 172.21.15.37:0/4067400822 learned_addr learned my addr 172.21.15.37:0/4067400822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec4d18640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec0101b60 0x7feec0103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5519640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec01011a0 0x7feec0102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 -- 172.21.15.37:0/4067400822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec0101b60 msgr2=0x7feec0103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec0101b60 0x7feec0103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 -- 172.21.15.37:0/4067400822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec01011a0 msgr2=0x7feec0102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec01011a0 0x7feec0102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.145+0000 7feec5d1a640 1 -- 172.21.15.37:0/4067400822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec01a6b30 con 0x7feec0106fb0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feec5d1a640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feeb800eb10 tx=0x7feeb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feeb27fc640 1 -- 172.21.15.37:0/4067400822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8004b20 con 0x7feec0106fb0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec01a6e20 con 0x7feec0106fb0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feeb27fc640 1 -- 172.21.15.37:0/4067400822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb8004cc0 con 0x7feec0106fb0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feeb27fc640 1 -- 172.21.15.37:0/4067400822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8010670 con 0x7feec0106fb0 2024-04-25T21:12:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.146+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01a72f0 con 0x7feec0106fb0 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.148+0000 7feeb27fc640 1 -- 172.21.15.37:0/4067400822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feeb8005020 con 0x7feec0106fb0 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 Processor -- start 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 -- start start 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a9034ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558a902b3970 con 0x558a90359800 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558a902504a0 con 0x558a9034e620 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeca571740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558a902af930 con 0x558a9034ef40 2024-04-25T21:12:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feeb27fc640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 0x7feeac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40204/0 (socket says 172.21.15.37:40204) 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec4d18640 1 -- 172.21.15.37:0/1750207628 learned_addr learned my addr 172.21.15.37:0/1750207628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec4d18640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a9034ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec5d1a640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 0x7feeac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.149+0000 7feec5519640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feeb27fc640 1 -- 172.21.15.37:0/4067400822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feeb80d0dc0 con 0x7feec0106fb0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 -- 172.21.15.37:0/1750207628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 msgr2=0x558a903592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 -- 172.21.15.37:0/1750207628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 msgr2=0x558a9034ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a9034ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 -- 172.21.15.37:0/1750207628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8ffc6210 con 0x558a90359800 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec5519640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec4d18640 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feeb4011810 tx=0x7feeb40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94891cce19622173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feec5d1a640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 0x7feeac0a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feeb809ca30 tx=0x7feeb806e770 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feeab7fe640 1 -- 172.21.15.37:0/1750207628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4004030 con 0x558a90359800 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feeab7fe640 1 -- 172.21.15.37:0/1750207628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb4011e50 con 0x558a90359800 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.150+0000 7feeab7fe640 1 -- 172.21.15.37:0/1750207628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb400fea0 con 0x558a90359800 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- 172.21.15.37:0/1750207628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 msgr2=0x558a9035bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7feeb4011810 tx=0x7feeb40097d0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- 172.21.15.37:0/1750207628 shutdown_connections 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a9035bbf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 --2- 172.21.15.37:0/1750207628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a9034ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- 172.21.15.37:0/1750207628 >> 172.21.15.37:0/1750207628 conn(0x558a9034ab30 msgr2=0x558a9034af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- 172.21.15.37:0/1750207628 shutdown_connections 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- 172.21.15.37:0/1750207628 wait complete. 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 -- start start 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.151+0000 7feeca571740 1 Processor -- start 2024-04-25T21:12:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a903590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558a9035c430 con 0x558a90359800 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558a9024e0b0 con 0x558a9034e620 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558a9035c5b0 con 0x558a9034ef40 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feeca571740 10 client.0 _ll_register_callbacks cb 0x558a9023bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40218/0 (socket says 172.21.15.37:40218) 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 -- 172.21.15.37:0/2950444124 learned_addr learned my addr 172.21.15.37:0/2950444124 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec5519640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a903590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 -- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 msgr2=0x558a903563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 -- 172.21.15.37:0/2950444124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 msgr2=0x558a903590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a903590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.152+0000 7feec4d18640 1 -- 172.21.15.37:0/2950444124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a90092620 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feec4d18640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feeb400fb80 tx=0x7feeb400fca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca006f309d2b708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb40051a0 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feeb4028040 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb4011e50 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeb4005340 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a90368310 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb4011e50 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feeb4004ad0 con 0x558a90359800 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:01.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 20 client.15801 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90369660 0x558a9036ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 4 client.15801 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.153+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a902c1390 con 0x558a90369660 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.154+0000 7feec5519640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90369660 0x558a9036ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.154+0000 7feec5519640 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90369660 0x558a9036ba50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9648fac0c96ba47d server_cookie=a529c0839b97a71e in_seq=0 out_seq=0 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.154+0000 7feea8ff9640 10 client.15801 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feea8ff9640 1 -- 172.21.15.37:0/2950444124 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a902c1390 con 0x558a90369660 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feea8ff9640 10 client.15801 handle_command_reply: tid=0 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 client.15801 shutdown 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 msgr2=0x558a90356910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7feeb400fb80 tx=0x7feeb400fca0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 shutdown_connections 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90369660 0x558a9036ba50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a90359800 0x558a90356910 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feea8ff9640 0 client.15801 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ef40 0x558a903563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 --2- 172.21.15.37:0/2950444124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034e620 0x558a903590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 >> 172.21.15.37:0/2950444124 conn(0x558a9034ab30 msgr2=0x558a90350860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 shutdown_connections 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 1 -- 172.21.15.37:0/2950444124 wait complete. 2024-04-25T21:12:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.155+0000 7feeca571740 20 client.15801 trim_cache size 0 max 16384 2024-04-25T21:12:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.170+0000 7feeca571740 1 Processor -- start 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.170+0000 7feeca571740 1 -- start start 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ab30 0x558a903652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9034a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feec01afc70 con 0x558a9034e620 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b17b0 con 0x558a9034ef40 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeca571740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b0340 con 0x558a9034ab30 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40230/0 (socket says 172.21.15.37:40230) 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 -- 172.21.15.37:0/1287164634 learned_addr learned my addr 172.21.15.37:0/1287164634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec4d18640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ab30 0x558a903652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 -- 172.21.15.37:0/1287164634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ab30 msgr2=0x558a903652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ab30 0x558a903652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec4d18640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9034a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 -- 172.21.15.37:0/1287164634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 msgr2=0x558a9034a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9034a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 -- 172.21.15.37:0/1287164634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a9036bdc0 con 0x558a9034e620 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec4d18640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9034a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feec5519640 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feebc005ab0 tx=0x7feebc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68062a3cb966ddef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeaaffd640 1 -- 172.21.15.37:0/1287164634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc049070 con 0x558a9034e620 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeaaffd640 1 -- 172.21.15.37:0/1287164634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc044dc0 con 0x558a9034e620 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.171+0000 7feeaaffd640 1 -- 172.21.15.37:0/1287164634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc035040 con 0x558a9034e620 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- 172.21.15.37:0/1287164634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 msgr2=0x558a90349b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feebc005ab0 tx=0x7feebc0092d0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- 172.21.15.37:0/1287164634 shutdown_connections 2024-04-25T21:12:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9034a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034e620 0x558a90349b30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 --2- 172.21.15.37:0/1287164634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034ab30 0x558a903652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- 172.21.15.37:0/1287164634 >> 172.21.15.37:0/1287164634 conn(0x558a90356a70 msgr2=0x558a90349420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- 172.21.15.37:0/1287164634 shutdown_connections 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- 172.21.15.37:0/1287164634 wait complete. 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 -- start start 2024-04-25T21:12:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.172+0000 7feeca571740 1 Processor -- start 2024-04-25T21:12:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034e620 0x558a9035ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9035f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b10e0 con 0x558a9034ab30 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feec01b0a10 con 0x558a9034ef40 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7feec01adc50 con 0x558a9034e620 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feeca571740 10 client.0 _ll_register_callbacks cb 0x7feeb4005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40234/0 (socket says 172.21.15.37:40234) 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 -- 172.21.15.37:0/3034711178 learned_addr learned my addr 172.21.15.37:0/3034711178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec5519640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034e620 0x558a9035ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 -- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034e620 msgr2=0x558a9035ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034e620 0x558a9035ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 -- 172.21.15.37:0/3034711178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 msgr2=0x558a9035f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9035f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.173+0000 7feec4d18640 1 -- 172.21.15.37:0/3034711178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a9035f7f0 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feec4d18640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x558a9034a5c0 tx=0x7feeb4009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e184e2efb08de131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4012e60 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7feeb4003730 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feebc044dc0 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeb40054e0 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 10 client.15807 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558a903cf760 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc044dc0 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feeb402d430 con 0x558a9034ab30 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 20 client.15807 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90359800 0x558a903d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 4 client.15807 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.174+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a902bb200 con 0x558a90359800 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.175+0000 7feec5519640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90359800 0x558a903d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.175+0000 7feea8ff9640 10 client.15807 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.175+0000 7feec5519640 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90359800 0x558a903d1f50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d0b928b115680be server_cookie=fb680df3eb3bb88c in_seq=0 out_seq=0 2024-04-25T21:12:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.188+0000 7feea8ff9640 1 -- 172.21.15.37:0/3034711178 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a902bb200 con 0x558a90359800 2024-04-25T21:12:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.188+0000 7feea8ff9640 10 client.15807 handle_command_reply: tid=0 2024-04-25T21:12:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.188+0000 7feeca571740 1 client.15807 shutdown 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 msgr2=0x558a903618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x558a9034a5c0 tx=0x7feeb4009e10 comp rx=0 tx=0).stop 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 shutdown_connections 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558a90359800 0x558a903d1f50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558a9034ef40 0x558a9035f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558a9034e620 0x558a9035ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 --2- 172.21.15.37:0/3034711178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558a9034ab30 0x558a903618b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feea8ff9640 0 client.15807 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 >> 172.21.15.37:0/3034711178 conn(0x558a90356a70 msgr2=0x558a90349cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 shutdown_connections 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 1 -- 172.21.15.37:0/3034711178 wait complete. 2024-04-25T21:12:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feeca571740 20 client.15807 trim_cache size 0 max 16384 2024-04-25T21:12:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:12:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:12:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:12:01.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 msgr2=0x7feeac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 0x7feeac0a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feeb809ca30 tx=0x7feeb806e770 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 msgr2=0x7feec0103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.189+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feeb800eb10 tx=0x7feeb800c2d0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 shutdown_connections 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feec0106fb0 0x7feec0103590 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7feeac0a3dc0 0x7feeac0a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feec0101b60 0x7feec0103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 --2- 172.21.15.37:0/4067400822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feec01011a0 0x7feec0102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 >> 172.21.15.37:0/4067400822 conn(0x7feec0076c50 msgr2=0x7feec006a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:01.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 shutdown_connections 2024-04-25T21:12:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:01.190+0000 7feec777d640 1 -- 172.21.15.37:0/4067400822 wait complete. 2024-04-25T21:12:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:01 smithi098 ceph-mon[32569]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:12:01.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:01 smithi037 ceph-mon[29195]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:12:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:01 smithi177 ceph-mon[33043]: pgmap v109: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-25T21:12:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:03 smithi098 ceph-mon[32569]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:12:03.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:03 smithi037 ceph-mon[29195]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:12:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:03 smithi177 ceph-mon[33043]: pgmap v110: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-25T21:12:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:05 smithi098 ceph-mon[32569]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 59 KiB/s wr, 37 op/s 2024-04-25T21:12:05.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:05 smithi037 ceph-mon[29195]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 59 KiB/s wr, 37 op/s 2024-04-25T21:12:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:05 smithi177 ceph-mon[33043]: pgmap v111: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 59 KiB/s wr, 37 op/s 2024-04-25T21:12:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:07 smithi098 ceph-mon[32569]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:07.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:07 smithi037 ceph-mon[29195]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:07 smithi177 ceph-mon[33043]: pgmap v112: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:09.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:09 smithi098 ceph-mon[32569]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:09.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:09 smithi037 ceph-mon[29195]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:09.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:09 smithi177 ceph-mon[33043]: pgmap v113: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:11.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:11 smithi037 ceph-mon[29195]: pgmap v114: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-25T21:12:11.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:11 smithi177 ceph-mon[33043]: pgmap v114: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-25T21:12:11.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:11 smithi098 ceph-mon[32569]: pgmap v114: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-25T21:12:13.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:13 smithi037 ceph-mon[29195]: pgmap v115: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:13.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:13 smithi177 ceph-mon[33043]: pgmap v115: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:13.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:13 smithi098 ceph-mon[32569]: pgmap v115: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 30 op/s 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 -- 172.21.15.37:0/3285595267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13301021c0 msgr2=0x7f13301025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 --2- 172.21.15.37:0/3285595267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13301021c0 0x7f13301025a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1324009900 tx=0x7f132402a3f0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 -- 172.21.15.37:0/3285595267 shutdown_connections 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 --2- 172.21.15.37:0/3285595267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1330107290 0x7f133010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 --2- 172.21.15.37:0/3285595267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330102b70 0x7f1330106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 --2- 172.21.15.37:0/3285595267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13301021c0 0x7f13301025a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 -- 172.21.15.37:0/3285595267 >> 172.21.15.37:0/3285595267 conn(0x7f13300fda20 msgr2=0x7f13300ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 -- 172.21.15.37:0/3285595267 shutdown_connections 2024-04-25T21:12:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.289+0000 7f1336d94640 1 -- 172.21.15.37:0/3285595267 wait complete. 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 Processor -- start 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 -- start start 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 0x7f1330074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330107290 0x7f1330074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13301121d0 con 0x7f1330074cd0 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1330112050 con 0x7f1330107290 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1336d94640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1330112350 con 0x7f13301021c0 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47166/0 (socket says 172.21.15.37:47166) 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.290+0000 7f1335331640 1 -- 172.21.15.37:0/2222160182 learned_addr learned my addr 172.21.15.37:0/2222160182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f132ffff640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330107290 0x7f1330074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1334b30640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 0x7f1330074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 -- 172.21.15.37:0/2222160182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 msgr2=0x7f1330074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 0x7f1330074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 -- 172.21.15.37:0/2222160182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330107290 msgr2=0x7f1330074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330107290 0x7f1330074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 -- 172.21.15.37:0/2222160182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301a6e80 con 0x7f1330074cd0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1334b30640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 0x7f1330074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1335331640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f132800cce0 tx=0x7f132800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f132dffb640 1 -- 172.21.15.37:0/2222160182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132800f450 con 0x7f1330074cd0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f132dffb640 1 -- 172.21.15.37:0/2222160182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132800faa0 con 0x7f1330074cd0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13301a7170 con 0x7f1330074cd0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f132dffb640 1 -- 172.21.15.37:0/2222160182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328018a80 con 0x7f1330074cd0 2024-04-25T21:12:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.291+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13301a7670 con 0x7f1330074cd0 2024-04-25T21:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f132dffb640 1 -- 172.21.15.37:0/2222160182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f132800fc50 con 0x7f1330074cd0 2024-04-25T21:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 Processor -- start 2024-04-25T21:12:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 -- start start 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb09ffaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f132dffb640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 0x7f13200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb0a005800 0x55eb0a007bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb09f5f970 con 0x55eb09ffaf40 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb09efc4a0 con 0x55eb09ffa620 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1339b89740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eb09f5b930 con 0x55eb0a005800 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f132ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.294+0000 7f1334b30640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 0x7f13200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47182/0 (socket says 172.21.15.37:47182) 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 -- 172.21.15.37:0/4228022704 learned_addr learned my addr 172.21.15.37:0/4228022704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132dffb640 1 -- 172.21.15.37:0/2222160182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f13280d0c00 con 0x7f1330074cd0 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f1335331640 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb0a005800 0x55eb0a007bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb09ffaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 -- 172.21.15.37:0/4228022704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb0a005800 msgr2=0x55eb0a007bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb0a005800 0x55eb0a007bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 -- 172.21.15.37:0/4228022704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 msgr2=0x55eb09ffaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb09ffaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 -- 172.21.15.37:0/4228022704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb09c72210 con 0x55eb09ffaf40 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f1334b30640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 0x7f13200a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1324031440 tx=0x7f13240095f0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f132ffff640 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f131c00afd0 tx=0x7f131c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f33ba891d71f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f131affd640 1 -- 172.21.15.37:0/4228022704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c005220 con 0x55eb09ffaf40 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f131affd640 1 -- 172.21.15.37:0/4228022704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f131c004c50 con 0x55eb09ffaf40 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.295+0000 7f131affd640 1 -- 172.21.15.37:0/4228022704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c00fbb0 con 0x55eb09ffaf40 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- 172.21.15.37:0/4228022704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 msgr2=0x55eb0a0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f131c00afd0 tx=0x7f131c00a7d0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- 172.21.15.37:0/4228022704 shutdown_connections 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb0a005800 0x55eb0a007bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb09ffaf40 0x55eb0a0052c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 --2- 172.21.15.37:0/4228022704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb09ffaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- 172.21.15.37:0/4228022704 >> 172.21.15.37:0/4228022704 conn(0x55eb09ff6b30 msgr2=0x55eb09ff6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- 172.21.15.37:0/4228022704 shutdown_connections 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- 172.21.15.37:0/4228022704 wait complete. 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 -- start start 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.296+0000 7f1339b89740 1 Processor -- start 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a002440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb0a008430 con 0x55eb0a005800 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55eb09efa0b0 con 0x55eb09ffa620 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55eb0a0085b0 con 0x55eb09ffaf40 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1339b89740 10 client.0 _ll_register_callbacks cb 0x55eb09ee7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47194/0 (socket says 172.21.15.37:47194) 2024-04-25T21:12:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 -- 172.21.15.37:0/3181313021 learned_addr learned my addr 172.21.15.37:0/3181313021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f132ffff640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a002440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1334b30640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 -- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 msgr2=0x55eb0a002440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a002440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 -- 172.21.15.37:0/3181313021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 msgr2=0x55eb0a005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.297+0000 7f1335331640 1 -- 172.21.15.37:0/3181313021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb09d3e620 con 0x55eb0a005800 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1335331640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f132809c8d0 tx=0x7f132809d2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab102cb939925522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132809cbd0 con 0x55eb0a005800 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f13280aa170 con 0x55eb0a005800 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c004c50 con 0x55eb0a005800 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1328094020 con 0x55eb0a005800 2024-04-25T21:12:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb0a014360 con 0x55eb0a005800 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f131c004c50 con 0x55eb0a005800 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1328020030 con 0x55eb0a005800 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 20 client.15819 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a0157d0 0x55eb0a017bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 4 client.15819 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.298+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb09f6d390 con 0x55eb0a0157d0 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.299+0000 7f1334b30640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a0157d0 0x55eb0a017bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.299+0000 7f1334b30640 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a0157d0 0x55eb0a017bc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1505c2fe07375f server_cookie=cfe8c735b73d5bb7 in_seq=0 out_seq=0 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.299+0000 7f130ffff640 10 client.15819 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f130ffff640 1 -- 172.21.15.37:0/3181313021 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb09f6d390 con 0x55eb0a0157d0 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f130ffff640 10 client.15819 handle_command_reply: tid=0 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 client.15819 shutdown 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 msgr2=0x55eb0a0029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f132809c8d0 tx=0x7f132809d2d0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 shutdown_connections 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a0157d0 0x55eb0a017bc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a0029a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f130ffff640 0 client.15819 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a002440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 --2- 172.21.15.37:0/3181313021 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 >> 172.21.15.37:0/3181313021 conn(0x55eb09ff6b30 msgr2=0x55eb09ffc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 shutdown_connections 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 1 -- 172.21.15.37:0/3181313021 wait complete. 2024-04-25T21:12:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.300+0000 7f1339b89740 20 client.15819 trim_cache size 0 max 16384 2024-04-25T21:12:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.315+0000 7f1339b89740 1 Processor -- start 2024-04-25T21:12:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.315+0000 7f1339b89740 1 -- start start 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.315+0000 7f1339b89740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a0112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.315+0000 7f1339b89740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a01d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.315+0000 7f1339b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1339b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13301afae0 con 0x55eb0a005800 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1339b89740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13301b1620 con 0x55eb09ffa620 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1339b89740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13301b01b0 con 0x55eb09ffaf40 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47204/0 (socket says 172.21.15.37:47204) 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 -- 172.21.15.37:0/2509754780 learned_addr learned my addr 172.21.15.37:0/2509754780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1334b30640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a0112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f132ffff640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a01d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 -- 172.21.15.37:0/2509754780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 msgr2=0x55eb0a01d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a01d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 -- 172.21.15.37:0/2509754780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 msgr2=0x55eb0a0112f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a0112f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 -- 172.21.15.37:0/2509754780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb09ff22e0 con 0x55eb0a005800 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1334b30640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a0112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f132ffff640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a01d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f1335331640 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1328098fe0 tx=0x7f13280b6240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489e989425ed27de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f131a7fc640 1 -- 172.21.15.37:0/2509754780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280a5840 con 0x55eb0a005800 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f131a7fc640 1 -- 172.21.15.37:0/2509754780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13280a0f80 con 0x55eb0a005800 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.316+0000 7f131a7fc640 1 -- 172.21.15.37:0/2509754780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132809c280 con 0x55eb0a005800 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- 172.21.15.37:0/2509754780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 msgr2=0x55eb0a01da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1328098fe0 tx=0x7f13280b6240 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- 172.21.15.37:0/2509754780 shutdown_connections 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a01da10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffaf40 0x55eb0a01d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 --2- 172.21.15.37:0/2509754780 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffa620 0x55eb0a0112f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- 172.21.15.37:0/2509754780 >> 172.21.15.37:0/2509754780 conn(0x55eb09ff6b30 msgr2=0x55eb0a002400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- 172.21.15.37:0/2509754780 shutdown_connections 2024-04-25T21:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- 172.21.15.37:0/2509754780 wait complete. 2024-04-25T21:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 -- start start 2024-04-25T21:12:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.317+0000 7f1339b89740 1 Processor -- start 2024-04-25T21:12:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffa620 0x55eb0a00b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffaf40 0x55eb0a00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13301b0f50 con 0x55eb0a005800 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13301b0880 con 0x55eb09ffaf40 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f13301adac0 con 0x55eb09ffa620 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1339b89740 10 client.0 _ll_register_callbacks cb 0x7f132809c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47210/0 (socket says 172.21.15.37:47210) 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 -- 172.21.15.37:0/1944780029 learned_addr learned my addr 172.21.15.37:0/1944780029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f132ffff640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffaf40 0x55eb0a00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1334b30640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffa620 0x55eb0a00b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 -- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffa620 msgr2=0x55eb0a00b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffa620 0x55eb0a00b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 -- 172.21.15.37:0/1944780029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffaf40 msgr2=0x55eb0a00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffaf40 0x55eb0a00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 -- 172.21.15.37:0/1944780029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0a017f30 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f1335331640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f132809ae80 tx=0x7f132809ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5418618ac92f977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.318+0000 7f130ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132809cbd0 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f132800f800 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13280a0f80 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f13280974d0 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb0a07bfc0 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13280a0f80 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1328020070 con 0x55eb0a005800 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 20 client.15825 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a07c3f0 0x55eb0a07e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 4 client.15825 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:12:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.319+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb09f67200 con 0x55eb0a07c3f0 2024-04-25T21:12:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.320+0000 7f1334b30640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a07c3f0 0x55eb0a07e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.320+0000 7f1334b30640 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a07c3f0 0x55eb0a07e7e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2777dcb3be96522b server_cookie=1a169b909c248754 in_seq=0 out_seq=0 2024-04-25T21:12:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.320+0000 7f130ffff640 10 client.15825 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f130ffff640 1 -- 172.21.15.37:0/1944780029 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb09f67200 con 0x55eb0a07c3f0 2024-04-25T21:12:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f130ffff640 10 client.15825 handle_command_reply: tid=0 2024-04-25T21:12:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 client.15825 shutdown 2024-04-25T21:12:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 msgr2=0x55eb0a00bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f132809ae80 tx=0x7f132809ca20 comp rx=0 tx=0).stop 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 shutdown_connections 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55eb0a07c3f0 0x55eb0a07e7e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0a005800 0x55eb0a00bed0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55eb09ffaf40 0x55eb0a00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f130ffff640 0 client.15825 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 --2- 172.21.15.37:0/1944780029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55eb09ffa620 0x55eb0a00b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 >> 172.21.15.37:0/1944780029 conn(0x55eb09ff6b30 msgr2=0x55eb0a003c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 shutdown_connections 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 1 -- 172.21.15.37:0/1944780029 wait complete. 2024-04-25T21:12:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.333+0000 7f1339b89740 20 client.15825 trim_cache size 0 max 16384 2024-04-25T21:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:12:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 msgr2=0x7f13200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 0x7f13200a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1324031440 tx=0x7f13240095f0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 msgr2=0x7f13301a67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f132800cce0 tx=0x7f132800a620 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 shutdown_connections 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1330074cd0 0x7f13301a67e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1330107290 0x7f1330074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f13200a3dc0 0x7f13200a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 --2- 172.21.15.37:0/2222160182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f13301021c0 0x7f1330074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 >> 172.21.15.37:0/2222160182 conn(0x7f13300fda20 msgr2=0x7f13300fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 shutdown_connections 2024-04-25T21:12:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:14.334+0000 7f1336d94640 1 -- 172.21.15.37:0/2222160182 wait complete. 2024-04-25T21:12:15.719 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:15 smithi098 ceph-mon[32569]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-04-25T21:12:15.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:15 smithi037 ceph-mon[29195]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-04-25T21:12:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:15 smithi177 ceph-mon[33043]: pgmap v116: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 62 KiB/s wr, 36 op/s 2024-04-25T21:12:17.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:17 smithi037 ceph-mon[29195]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:17.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:17 smithi177 ceph-mon[33043]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:17 smithi098 ceph-mon[32569]: pgmap v117: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:19.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:19 smithi037 ceph-mon[29195]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:19.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:19 smithi177 ceph-mon[33043]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:19.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:19 smithi098 ceph-mon[32569]: pgmap v118: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2024-04-25T21:12:21.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:21 smithi037 ceph-mon[29195]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-25T21:12:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:21 smithi177 ceph-mon[33043]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-25T21:12:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:21 smithi098 ceph-mon[32569]: pgmap v119: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 21 op/s 2024-04-25T21:12:23.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:23 smithi037 ceph-mon[29195]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:12:23.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:23 smithi177 ceph-mon[33043]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:12:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:23 smithi098 ceph-mon[32569]: pgmap v120: 97 pgs: 97 active+clean; 3.7 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:12:24.573 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:12:24.574 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.8 2024-04-25T21:12:24.574 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:12:24.836 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:12:25.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:25 smithi037 ceph-mon[29195]: pgmap v121: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-25T21:12:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:25 smithi177 ceph-mon[33043]: pgmap v121: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-25T21:12:25.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:25 smithi098 ceph-mon[32569]: pgmap v121: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.727+0000 7f699939d700 1 -- 172.21.15.37:0/4141157797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406bbb0 msgr2=0x7f699410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.727+0000 7f699939d700 1 --2- 172.21.15.37:0/4141157797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406bbb0 0x7f699410f930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f698c00b0a0 tx=0x7f698c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 -- 172.21.15.37:0/4141157797 shutdown_connections 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 --2- 172.21.15.37:0/4141157797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406bbb0 0x7f699410f930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 --2- 172.21.15.37:0/4141157797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 --2- 172.21.15.37:0/4141157797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406b1f0 0x7f699406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 -- 172.21.15.37:0/4141157797 >> 172.21.15.37:0/4141157797 conn(0x7f69940789e0 msgr2=0x7f6994078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 -- 172.21.15.37:0/4141157797 shutdown_connections 2024-04-25T21:12:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 -- 172.21.15.37:0/4141157797 wait complete. 2024-04-25T21:12:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 Processor -- start 2024-04-25T21:12:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.728+0000 7f699939d700 1 -- start start 2024-04-25T21:12:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406bbb0 0x7f6994101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:26.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:26.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6994112c40 con 0x7f699406b1f0 2024-04-25T21:12:26.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6994112ac0 con 0x7f6994108a60 2024-04-25T21:12:26.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f699939d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6994112dc0 con 0x7f699406bbb0 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49288/0 (socket says 172.21.15.37:49288) 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 -- 172.21.15.37:0/849104593 learned_addr learned my addr 172.21.15.37:0/849104593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f69927fc700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406bbb0 0x7f6994101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f69937fe700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:26.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 -- 172.21.15.37:0/849104593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406bbb0 msgr2=0x7f6994101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406bbb0 0x7f6994101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 -- 172.21.15.37:0/849104593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 msgr2=0x7f6994102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 -- 172.21.15.37:0/849104593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69941a7620 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f69937fe700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f6992ffd700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f698400d980 tx=0x7f698400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.729+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984016070 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.730+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69941a7910 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.730+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984004830 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.730+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840054c0 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.730+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69941a7ea0 con 0x7f699406b1f0 2024-04-25T21:12:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.730+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69941041a0 con 0x7f699406b1f0 2024-04-25T21:12:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.732+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6984004220 con 0x7f699406b1f0 2024-04-25T21:12:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.732+0000 7f697bfff700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 0x7f697c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.732+0000 7f69927fc700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 0x7f697c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.732+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f69840d0a40 con 0x7f699406b1f0 2024-04-25T21:12:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.733+0000 7f69927fc700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 0x7f697c0a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6994078ee0 tx=0x7f698800b410 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.736+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f698401f020 con 0x7f699406b1f0 2024-04-25T21:12:26.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.993+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f699402df80 con 0x7f699406b1f0 2024-04-25T21:12:26.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.994+0000 7f697bfff700 1 -- 172.21.15.37:0/849104593 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f69840193b0 con 0x7f699406b1f0 2024-04-25T21:12:26.994 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:12:26.995 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:12:26.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 msgr2=0x7f697c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 0x7f697c0a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6994078ee0 tx=0x7f698800b410 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 msgr2=0x7f69941019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f698400d980 tx=0x7f698400dc90 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 shutdown_connections 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f697c0a3db0 0x7f697c0a6270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6994108a60 0x7f6994102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f699406b1f0 0x7f69941019b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 --2- 172.21.15.37:0/849104593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f699406bbb0 0x7f6994101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 >> 172.21.15.37:0/849104593 conn(0x7f69940789e0 msgr2=0x7f69940ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 shutdown_connections 2024-04-25T21:12:26.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:26.996+0000 7f699939d700 1 -- 172.21.15.37:0/849104593 wait complete. 2024-04-25T21:12:26.998 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 -- 172.21.15.37:0/467985141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 msgr2=0x7fb0b410c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/467985141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b410c020 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009900 tx=0x7fb0a802a480 comp rx=0 tx=0).stop 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 -- 172.21.15.37:0/467985141 shutdown_connections 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/467985141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b410c560 0x7fb0b410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/467985141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b410c020 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/467985141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b4103220 0x7fb0b4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 -- 172.21.15.37:0/467985141 >> 172.21.15.37:0/467985141 conn(0x7fb0b4077fd0 msgr2=0x7fb0b40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 -- 172.21.15.37:0/467985141 shutdown_connections 2024-04-25T21:12:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 -- 172.21.15.37:0/467985141 wait complete. 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.427+0000 7fb0b97f7640 1 Processor -- start 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 -- start start 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b4103220 0x7fb0b4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 0x7fb0b419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112200 con 0x7fb0b4103b40 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112080 con 0x7fb0b410c560 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b97f7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b4112380 con 0x7fb0b4103220 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49304/0 (socket says 172.21.15.37:49304) 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 -- 172.21.15.37:0/843622024 learned_addr learned my addr 172.21.15.37:0/843622024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 0x7fb0b419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b4103220 0x7fb0b4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 -- 172.21.15.37:0/843622024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b4103220 msgr2=0x7fb0b4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b4103220 0x7fb0b4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 -- 172.21.15.37:0/843622024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 msgr2=0x7fb0b419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 0x7fb0b419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b27fc640 1 -- 172.21.15.37:0/843622024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b419d670 con 0x7fb0b4103b40 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.428+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 0x7fb0b419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005ab0 tx=0x7fb0a8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb09bfff640 1 -- 172.21.15.37:0/843622024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a803f070 con 0x7fb0b4103b40 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb09bfff640 1 -- 172.21.15.37:0/843622024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a8031d60 con 0x7fb0b4103b40 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb09bfff640 1 -- 172.21.15.37:0/843622024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8035030 con 0x7fb0b4103b40 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b419d900 con 0x7fb0b4103b40 2024-04-25T21:12:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.429+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b41a51f0 con 0x7fb0b4103b40 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 Processor -- start 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb09bfff640 1 -- 172.21.15.37:0/843622024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb0a8042070 con 0x7fb0b4103b40 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 -- start start 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fadebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567fad43970 con 0x5567fadde620 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5567face04a0 con 0x5567fade9800 2024-04-25T21:12:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5567fad3f930 con 0x5567faddef40 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49318/0 (socket says 172.21.15.37:49318) 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 -- 172.21.15.37:0/322188024 learned_addr learned my addr 172.21.15.37:0/322188024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fadebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 -- 172.21.15.37:0/322188024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 msgr2=0x5567fade92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 -- 172.21.15.37:0/322188024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 msgr2=0x5567fadebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fadebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb0b37fe640 1 -- 172.21.15.37:0/322188024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567faa56210 con 0x5567fadde620 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.432+0000 7fb09bfff640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 0x7fb09c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 0x7fb09c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009fd0 tx=0x7fb0ac010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09a2cd3f1853a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb09bfff640 1 -- 172.21.15.37:0/843622024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb0a80f1000 con 0x7fb0b4103b40 2024-04-25T21:12:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb0997fa640 1 -- 172.21.15.37:0/322188024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011700 con 0x5567fadde620 2024-04-25T21:12:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb0997fa640 1 -- 172.21.15.37:0/322188024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac011d50 con 0x5567fadde620 2024-04-25T21:12:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.433+0000 7fb0997fa640 1 -- 172.21.15.37:0/322188024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00e3c0 con 0x5567fadde620 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/322188024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 msgr2=0x5567faddea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 0x7fb09c0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40058c0 tx=0x7fb0a4005850 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009fd0 tx=0x7fb0ac010860 comp rx=0 tx=0).stop 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/322188024 shutdown_connections 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fadebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/322188024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567faddea00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/322188024 >> 172.21.15.37:0/322188024 conn(0x5567faddab30 msgr2=0x5567faddaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/322188024 shutdown_connections 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/322188024 wait complete. 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- start start 2024-04-25T21:12:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 Processor -- start 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fadde620 0x5567fade2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fade30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567fadec370 con 0x5567faddef40 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5567facde0b0 con 0x5567fade9800 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5567fadec4f0 con 0x5567fadde620 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.434+0000 7fb0bc5e6740 10 client.0 _ll_register_callbacks cb 0x5567faccbd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49324/0 (socket says 172.21.15.37:49324) 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 -- 172.21.15.37:0/2417739917 learned_addr learned my addr 172.21.15.37:0/2417739917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fade30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fadde620 0x5567fade2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 -- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fadde620 msgr2=0x5567fade2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fadde620 0x5567fade2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 -- 172.21.15.37:0/2417739917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 msgr2=0x5567fade30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fade30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 -- 172.21.15.37:0/2417739917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fab22620 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fade30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8031440 tx=0x7fb0a8031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67504ad1f3baff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb096ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8049040 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb0a80f3110 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0ac011d50 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a80b9270 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0bc5e6740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.435+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567fae00b70 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ac011d50 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb0a80eea00 con 0x5567faddef40 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 20 client.15843 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae02070 0x5567fae04460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 4 client.15843 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567fad51390 con 0x5567fae02070 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.436+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae02070 0x5567fae04460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.437+0000 7fb096ff5640 10 client.15843 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.437+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae02070 0x5567fae04460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f043afe796981801 server_cookie=6668848c960be767 in_seq=0 out_seq=0 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.437+0000 7fb096ff5640 1 -- 172.21.15.37:0/2417739917 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567fad51390 con 0x5567fae02070 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.437+0000 7fb096ff5640 10 client.15843 handle_command_reply: tid=0 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.437+0000 7fb0bc5e6740 1 client.15843 shutdown 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 msgr2=0x5567fade2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8031440 tx=0x7fb0a8031f10 comp rx=0 tx=0).stop 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 shutdown_connections 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae02070 0x5567fae04460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fade30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fade2ba0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2417739917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fadde620 0x5567fade2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 >> 172.21.15.37:0/2417739917 conn(0x5567faddab30 msgr2=0x5567fade0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb096ff5640 0 client.15843 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 shutdown_connections 2024-04-25T21:12:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2417739917 wait complete. 2024-04-25T21:12:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.438+0000 7fb0bc5e6740 20 client.15843 trim_cache size 0 max 16384 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 Processor -- start 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 -- start start 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fae09d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41ab850 con 0x5567fadde620 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41ad390 con 0x5567fade9800 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41abf20 con 0x5567faddef40 2024-04-25T21:12:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 -- 172.21.15.37:0/2985224295 learned_addr learned my addr 172.21.15.37:0/2985224295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fae09d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 -- 172.21.15.37:0/2985224295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 msgr2=0x5567fade2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 -- 172.21.15.37:0/2985224295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 msgr2=0x5567fae09d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fae09d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.456+0000 7fb0b2ffd640 1 -- 172.21.15.37:0/2985224295 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567fae04a40 con 0x5567fadde620 2024-04-25T21:12:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.457+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5567faddfbc0 tx=0x7fb0a4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18daef7a37d5440f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.457+0000 7fb098ff9640 1 -- 172.21.15.37:0/2985224295 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a402e070 con 0x5567fadde620 2024-04-25T21:12:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.457+0000 7fb098ff9640 1 -- 172.21.15.37:0/2985224295 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a4011ce0 con 0x5567fadde620 2024-04-25T21:12:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.457+0000 7fb098ff9640 1 -- 172.21.15.37:0/2985224295 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4003850 con 0x5567fadde620 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2985224295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 msgr2=0x5567fade8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5567faddfbc0 tx=0x7fb0a4013000 comp rx=0 tx=0).stop 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2985224295 shutdown_connections 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fade9800 0x5567fae09d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567faddef40 0x5567fade2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/2985224295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567fadde620 0x5567fade8dd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2985224295 >> 172.21.15.37:0/2985224295 conn(0x5567faddab30 msgr2=0x5567fadecf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2985224295 shutdown_connections 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/2985224295 wait complete. 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- start start 2024-04-25T21:12:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 Processor -- start 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fadde620 0x5567fadfe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fade9800 0x5567fadf9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41accc0 con 0x5567faddef40 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41ac5f0 con 0x5567fadde620 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b41a9850 con 0x5567fade9800 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0bc5e6740 10 client.0 _ll_register_callbacks cb 0x7fb0a8003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49336/0 (socket says 172.21.15.37:49336) 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.458+0000 7fb0b27fc640 1 -- 172.21.15.37:0/1962448050 learned_addr learned my addr 172.21.15.37:0/1962448050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fadde620 0x5567fadfe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b37fe640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fade9800 0x5567fadf9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 -- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fade9800 msgr2=0x5567fadf9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fade9800 0x5567fadf9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 -- 172.21.15.37:0/1962448050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fadde620 msgr2=0x5567fadfe390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fadde620 0x5567fadfe390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 -- 172.21.15.37:0/1962448050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fadfa0d0 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0b27fc640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80c1540 tx=0x7fb0a80c5d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a013b3185b93a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb096ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8049040 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb0a80c59b0 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a4011ce0 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0a80bc930 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0bc5e6740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567fae681e0 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.459+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a4011ce0 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb0a80bc420 con 0x5567faddef40 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 20 client.15849 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae68820 0x5567fae6ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 4 client.15849 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:12:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567fad4b200 con 0x5567fae68820 2024-04-25T21:12:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae68820 0x5567fae6ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb0b2ffd640 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae68820 0x5567fae6ac40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14d0f32af1105bf server_cookie=b742c854a5d430be in_seq=0 out_seq=0 2024-04-25T21:12:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.460+0000 7fb096ff5640 10 client.15849 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb096ff5640 1 -- 172.21.15.37:0/1962448050 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567fad4b200 con 0x5567fae68820 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb096ff5640 10 client.15849 handle_command_reply: tid=0 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 client.15849 shutdown 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 msgr2=0x5567fadfe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80c1540 tx=0x7fb0a80c5d50 comp rx=0 tx=0).stop 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 shutdown_connections 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5567fae68820 0x5567fae6ac40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5567fade9800 0x5567fadf9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb096ff5640 0 client.15849 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567faddef40 0x5567fadfe8d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 --2- 172.21.15.37:0/1962448050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5567fadde620 0x5567fadfe390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 >> 172.21.15.37:0/1962448050 conn(0x5567faddab30 msgr2=0x5567fade5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 shutdown_connections 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 1 -- 172.21.15.37:0/1962448050 wait complete. 2024-04-25T21:12:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.475+0000 7fb0bc5e6740 20 client.15849 trim_cache size 0 max 16384 2024-04-25T21:12:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:12:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:12:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:12:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 msgr2=0x7fb09c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 0x7fb09c0a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40058c0 tx=0x7fb0a4005850 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 msgr2=0x7fb0b4198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8005ab0 tx=0x7fb0a8003c80 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 shutdown_connections 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0b410c560 0x7fb0b419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0b4103b40 0x7fb0b4198bc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb09c0a3e40 0x7fb09c0a6300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 --2- 172.21.15.37:0/843622024 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0b4103220 0x7fb0b4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 >> 172.21.15.37:0/843622024 conn(0x7fb0b4077fd0 msgr2=0x7fb0b40fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:27.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 shutdown_connections 2024-04-25T21:12:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:27.476+0000 7fb0b97f7640 1 -- 172.21.15.37:0/843622024 wait complete. 2024-04-25T21:12:27.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:27 smithi037 ceph-mon[29195]: pgmap v122: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:27.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:27 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/849104593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:27.740 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 84.8 --await-for 42.0' 2024-04-25T21:12:27.740 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 84.8 --await-for 42.0 2024-04-25T21:12:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:27 smithi177 ceph-mon[33043]: pgmap v122: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:27 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/849104593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:27 smithi098 ceph-mon[32569]: pgmap v122: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:27 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/849104593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:27.989 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:12:29.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:29 smithi037 ceph-mon[29195]: pgmap v123: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- 172.21.15.37:0/1799480476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc106930 msgr2=0x7f9ddc106d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 --2- 172.21.15.37:0/1799480476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc106930 0x7f9ddc106d10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009990 tx=0x7f9dd8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- 172.21.15.37:0/1799480476 shutdown_connections 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 --2- 172.21.15.37:0/1799480476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc102650 0x7f9ddc10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 --2- 172.21.15.37:0/1799480476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc106930 0x7f9ddc106d10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 --2- 172.21.15.37:0/1799480476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc101c90 0x7f9ddc102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- 172.21.15.37:0/1799480476 >> 172.21.15.37:0/1799480476 conn(0x7f9ddc0777b0 msgr2=0x7f9ddc077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- 172.21.15.37:0/1799480476 shutdown_connections 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- 172.21.15.37:0/1799480476 wait complete. 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 Processor -- start 2024-04-25T21:12:29.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.809+0000 7f9de3891700 1 -- start start 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc101c90 0x7f9ddc1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 0x7f9ddc19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc111b80 con 0x7f9ddc102650 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc111a00 con 0x7f9ddc106930 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de3891700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc111d00 con 0x7f9ddc101c90 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51362/0 (socket says 172.21.15.37:51362) 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 -- 172.21.15.37:0/206258153 learned_addr learned my addr 172.21.15.37:0/206258153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de162d700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc101c90 0x7f9ddc1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 0x7f9ddc19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 -- 172.21.15.37:0/206258153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc101c90 msgr2=0x7f9ddc1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc101c90 0x7f9ddc1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 -- 172.21.15.37:0/206258153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 msgr2=0x7f9ddc19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 0x7f9ddc19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 -- 172.21.15.37:0/206258153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc19e0b0 con 0x7f9ddc102650 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 0x7f9ddc19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00b530 tx=0x7f9dcc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.810+0000 7f9dd27fc700 1 -- 172.21.15.37:0/206258153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc015020 con 0x7f9ddc102650 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.811+0000 7f9dd27fc700 1 -- 172.21.15.37:0/206258153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dcc011ce0 con 0x7f9ddc102650 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.811+0000 7f9dd27fc700 1 -- 172.21.15.37:0/206258153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc010d20 con 0x7f9ddc102650 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.811+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc19e3a0 con 0x7f9ddc102650 2024-04-25T21:12:29.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.811+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a5ca0 con 0x7f9ddc102650 2024-04-25T21:12:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.813+0000 7f9dd27fc700 1 -- 172.21.15.37:0/206258153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9dcc018020 con 0x7f9ddc102650 2024-04-25T21:12:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.813+0000 7f9dd27fc700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 0x7f9dc80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.814+0000 7f9dd27fc700 1 -- 172.21.15.37:0/206258153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9dcc0cfeb0 con 0x7f9ddc102650 2024-04-25T21:12:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.814+0000 7f9de162d700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 0x7f9dc80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.814+0000 7f9de162d700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 0x7f9dc80a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005b40 tx=0x7f9dd8005a90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.814+0000 7f9dee11ab80 1 Processor -- start 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.814+0000 7f9dee11ab80 1 -- start start 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x56209378bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378c000 0x5620937969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5620936dee20 con 0x562093796f00 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562093685ac0 con 0x56209378c000 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5620936dade0 con 0x56209378b6e0 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51372/0 (socket says 172.21.15.37:51372) 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 -- 172.21.15.37:0/834642802 learned_addr learned my addr 172.21.15.37:0/834642802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:29.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378c000 0x5620937969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de162d700 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x56209378bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 -- 172.21.15.37:0/834642802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 msgr2=0x56209378bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x56209378bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 -- 172.21.15.37:0/834642802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378c000 msgr2=0x5620937969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378c000 0x5620937969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 -- 172.21.15.37:0/834642802 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620936498d0 con 0x562093796f00 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009d50 tx=0x7f9dd40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad306a1bd88d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dc77fe700 1 -- 172.21.15.37:0/834642802 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4019070 con 0x562093796f00 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.815+0000 7f9dc77fe700 1 -- 172.21.15.37:0/834642802 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4014ab0 con 0x562093796f00 2024-04-25T21:12:29.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dc77fe700 1 -- 172.21.15.37:0/834642802 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4013a80 con 0x562093796f00 2024-04-25T21:12:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 -- 172.21.15.37:0/834642802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 msgr2=0x5620937992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009d50 tx=0x7f9dd40077e0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 -- 172.21.15.37:0/834642802 shutdown_connections 2024-04-25T21:12:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x5620937992f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x56209378bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/834642802 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378c000 0x5620937969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 -- 172.21.15.37:0/834642802 >> 172.21.15.37:0/834642802 conn(0x5620936b8a20 msgr2=0x562093787ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 -- 172.21.15.37:0/834642802 shutdown_connections 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.816+0000 7f9dee11ab80 1 -- 172.21.15.37:0/834642802 wait complete. 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 -- start start 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 Processor -- start 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x5620937966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x562093793ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562093799990 con 0x56209378c000 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562093698740 con 0x562093796f00 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562093799b10 con 0x56209378b6e0 2024-04-25T21:12:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9dee11ab80 10 client.0 _ll_register_callbacks cb 0x562093677ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9de0e2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9de162d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x5620937966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9de0e2c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51386/0 (socket says 172.21.15.37:51386) 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.817+0000 7f9de0e2c700 1 -- 172.21.15.37:0/3842585685 learned_addr learned my addr 172.21.15.37:0/3842585685 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x562093793ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 -- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 msgr2=0x5620937966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x5620937966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 -- 172.21.15.37:0/3842585685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 msgr2=0x562093793ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x562093793ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 -- 172.21.15.37:0/3842585685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209362ca60 con 0x56209378c000 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc09bf20 tx=0x7f9dcc09c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f47237c51bb5600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0b4020 con 0x56209378c000 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9dcc09c3f0 con 0x56209378c000 2024-04-25T21:12:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd4014ab0 con 0x56209378c000 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc0a4c00 con 0x56209378c000 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dee11ab80 10 client.15861 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.818+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620937a4260 con 0x56209378c000 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd4014ab0 con 0x56209378c000 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9dcc09b3e0 con 0x56209378c000 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 10 client.15861 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 10 client.15861 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 20 client.15861 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x5620937a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 4 client.15861 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620936ec610 con 0x5620937a6780 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.819+0000 7f9de162d700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x5620937a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.820+0000 7f9dc4ff9700 10 client.15861 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:29.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.820+0000 7f9de162d700 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x5620937a8b70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fd6bfd4b866282 server_cookie=184faea6cd684bdd in_seq=0 out_seq=0 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3842585685 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620936ec610 con 0x5620937a6780 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dc4ff9700 10 client.15861 handle_command_reply: tid=0 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 client.15861 shutdown 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 msgr2=0x5620937939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc09bf20 tx=0x7f9dcc09c780 comp rx=0 tx=0).stop 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 shutdown_connections 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x5620937a8b70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dc4ff9700 0 client.15861 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x562093793ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378b6e0 0x5620937966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3842585685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378c000 0x5620937939a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 >> 172.21.15.37:0/3842585685 conn(0x5620936b8a20 msgr2=0x56209378e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 shutdown_connections 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3842585685 wait complete. 2024-04-25T21:12:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.821+0000 7f9dee11ab80 20 client.15861 trim_cache size 0 max 16384 2024-04-25T21:12:29.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:29 smithi177 ceph-mon[33043]: pgmap v123: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 Processor -- start 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 -- start start 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x56209379f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1ad290 con 0x56209378b6e0 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1aedd0 con 0x562093796f00 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1ad960 con 0x56209378c000 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51394/0 (socket says 172.21.15.37:51394) 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 -- 172.21.15.37:0/3192797744 learned_addr learned my addr 172.21.15.37:0/3192797744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x56209379f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 -- 172.21.15.37:0/3192797744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 msgr2=0x56209379ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 -- 172.21.15.37:0/3192797744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 msgr2=0x56209379f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x56209379f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.841+0000 7f9de162d700 1 -- 172.21.15.37:0/3192797744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56209378d140 con 0x56209378b6e0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9de162d700 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009990 tx=0x7f9dd8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81cabea844f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9dc6ffd700 1 -- 172.21.15.37:0/3192797744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80352d0 con 0x56209378b6e0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9dc6ffd700 1 -- 172.21.15.37:0/3192797744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd803a070 con 0x56209378b6e0 2024-04-25T21:12:29.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9dc6ffd700 1 -- 172.21.15.37:0/3192797744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800fb00 con 0x56209378b6e0 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3192797744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 msgr2=0x5620937961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.842+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8009990 tx=0x7f9dd8022000 comp rx=0 tx=0).stop 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3192797744 shutdown_connections 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562093796f00 0x56209379f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56209378b6e0 0x5620937961e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3192797744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3192797744 >> 172.21.15.37:0/3192797744 conn(0x5620936b8a20 msgr2=0x562093786e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3192797744 shutdown_connections 2024-04-25T21:12:29.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3192797744 wait complete. 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- start start 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 Processor -- start 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378b6e0 0x562093799d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1ae700 con 0x562093796f00 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1ae030 con 0x56209378b6e0 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.843+0000 7f9dee11ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc1ab230 con 0x56209378c000 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dee11ab80 10 client.0 _ll_register_callbacks cb 0x562093677ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51406/0 (socket says 172.21.15.37:51406) 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 -- 172.21.15.37:0/3178749761 learned_addr learned my addr 172.21.15.37:0/3178749761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de162d700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378b6e0 0x562093799d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de0e2c700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 -- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 msgr2=0x56209379a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 -- 172.21.15.37:0/3178749761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378b6e0 msgr2=0x562093799d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378b6e0 0x562093799d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 -- 172.21.15.37:0/3178749761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620938113a0 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9de1e2e700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40135f0 tx=0x7f9dd4014630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a439f5e52197f8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4019040 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f9dd4014a50 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd803a070 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9dd40090b0 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.844+0000 7f9dee11ab80 10 client.15867 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562093812680 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dd803a070 con 0x562093796f00 2024-04-25T21:12:29.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9dd4030450 con 0x562093796f00 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 10 client.15867 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 20 client.15867 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x562093814b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 4 client.15867 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.8, "await_for": 42.0}] 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.8, "await_for": 42.0}) -- 0x5620936e66b0 con 0x5620937a6780 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.845+0000 7f9de162d700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x562093814b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.846+0000 7f9dc4ff9700 10 client.15867 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:29.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.846+0000 7f9de162d700 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x562093814b70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ca0220ef7b94d6a server_cookie=e0febbb263b21060 in_seq=0 out_seq=0 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dc4ff9700 1 -- 172.21.15.37:0/3178749761 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5620936e66b0 con 0x5620937a6780 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dc4ff9700 10 client.15867 handle_command_reply: tid=0 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 client.15867 shutdown 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 msgr2=0x56209379a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40135f0 tx=0x7f9dd4014630 comp rx=0 tx=0).stop 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 shutdown_connections 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5620937a6780 0x562093814b70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562093796f00 0x56209379a800 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56209378b6e0 0x562093799d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 --2- 172.21.15.37:0/3178749761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56209378c000 0x56209379a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 >> 172.21.15.37:0/3178749761 conn(0x5620936b8a20 msgr2=0x5620937a2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dc4ff9700 0 client.15867 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 shutdown_connections 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 1 -- 172.21.15.37:0/3178749761 wait complete. 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.861+0000 7f9dee11ab80 20 client.15867 trim_cache size 0 max 16384 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "cd372993": { 2024-04-25T21:12:29.863 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.8, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:12:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:12:29.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 msgr2=0x7f9dc80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 0x7f9dc80a6390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8005b40 tx=0x7f9dd8005a90 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 msgr2=0x7f9ddc1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00b530 tx=0x7f9dcc00b8f0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 shutdown_connections 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9dc80a3ed0 0x7f9dc80a6390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ddc106930 0x7f9ddc19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ddc101c90 0x7f9ddc1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 --2- 172.21.15.37:0/206258153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ddc102650 0x7f9ddc1995e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 >> 172.21.15.37:0/206258153 conn(0x7f9ddc0777b0 msgr2=0x7f9ddc06b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 shutdown_connections 2024-04-25T21:12:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:29.862+0000 7f9de3891700 1 -- 172.21.15.37:0/206258153 wait complete. 2024-04-25T21:12:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:29 smithi098 ceph-mon[32569]: pgmap v123: 97 pgs: 97 active+clean; 3.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-25T21:12:30.664 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cd372993', quiesce took 6.1 seconds. Will release after: 24.8 2024-04-25T21:12:31.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:31 smithi037 ceph-mon[29195]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-25T21:12:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:31 smithi177 ceph-mon[33043]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-25T21:12:31.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:31 smithi098 ceph-mon[32569]: pgmap v124: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 24 op/s 2024-04-25T21:12:33.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:33 smithi037 ceph-mon[29195]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-25T21:12:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:33 smithi177 ceph-mon[33043]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-25T21:12:33.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:33 smithi098 ceph-mon[32569]: pgmap v125: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-25T21:12:35.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:35 smithi037 ceph-mon[29195]: pgmap v126: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-25T21:12:35.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:35 smithi177 ceph-mon[33043]: pgmap v126: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-25T21:12:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:35 smithi098 ceph-mon[32569]: pgmap v126: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-25T21:12:37.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:37 smithi037 ceph-mon[29195]: pgmap v127: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:37.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:37 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:12:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:37 smithi177 ceph-mon[33043]: pgmap v127: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:37 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:12:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:37 smithi098 ceph-mon[32569]: pgmap v127: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:37 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:12:38.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:12:38.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:12:38.737 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:12:38.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:12:38.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:12:38.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:12:38.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:12:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:12:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:12:39.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:39 smithi098 ceph-mon[32569]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:39 smithi037 ceph-mon[29195]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:39 smithi177 ceph-mon[33043]: pgmap v128: 97 pgs: 97 active+clean; 4.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 -- 172.21.15.37:0/3554338345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15141010e0 msgr2=0x7f15141014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 --2- 172.21.15.37:0/3554338345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15141010e0 0x7f15141014c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1504009900 tx=0x7f150402a410 comp rx=0 tx=0).stop 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 -- 172.21.15.37:0/3554338345 shutdown_connections 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 --2- 172.21.15.37:0/3554338345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 0x7f151410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 --2- 172.21.15.37:0/3554338345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1514101a90 0x7f1514105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 --2- 172.21.15.37:0/3554338345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15141010e0 0x7f15141014c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.575+0000 7f1519acf640 1 -- 172.21.15.37:0/3554338345 >> 172.21.15.37:0/3554338345 conn(0x7f15140771d0 msgr2=0x7f15140775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.576+0000 7f1519acf640 1 -- 172.21.15.37:0/3554338345 shutdown_connections 2024-04-25T21:12:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.576+0000 7f1519acf640 1 -- 172.21.15.37:0/3554338345 wait complete. 2024-04-25T21:12:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.576+0000 7f1519acf640 1 Processor -- start 2024-04-25T21:12:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.576+0000 7f1519acf640 1 -- start start 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15141010e0 0x7f15141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 0x7f1514198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111120 con 0x7f151419cf70 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1514110fa0 con 0x7f15141010e0 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1519acf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15141112a0 con 0x7f15141061b0 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35674/0 (socket says 172.21.15.37:35674) 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 -- 172.21.15.37:0/208456683 learned_addr learned my addr 172.21.15.37:0/208456683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f15137fe640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15141010e0 0x7f15141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1512ffd640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 0x7f1514198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 -- 172.21.15.37:0/208456683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 msgr2=0x7f1514198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 0x7f1514198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 -- 172.21.15.37:0/208456683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15141010e0 msgr2=0x7f15141986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15141010e0 0x7f15141986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 -- 172.21.15.37:0/208456683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151419da60 con 0x7f151419cf70 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1513fff640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f150c00eaa0 tx=0x7f150c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.577+0000 7f1510ff9640 1 -- 172.21.15.37:0/208456683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00cdb0 con 0x7f151419cf70 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.578+0000 7f1510ff9640 1 -- 172.21.15.37:0/208456683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c005020 con 0x7f151419cf70 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.578+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151419dd50 con 0x7f151419cf70 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.578+0000 7f1510ff9640 1 -- 172.21.15.37:0/208456683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0116d0 con 0x7f151419cf70 2024-04-25T21:12:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.578+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a5540 con 0x7f151419cf70 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f1510ff9640 1 -- 172.21.15.37:0/208456683 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f150c0051d0 con 0x7f151419cf70 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 Processor -- start 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 -- start start 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7cda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cdf40 0x55c85c7d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c85c732970 con 0x55c85c7d8800 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c85c6cf4a0 con 0x55c85c7cdf40 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.580+0000 7f151c7e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c85c72e930 con 0x55c85c7cd620 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1510ff9640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 0x7f14fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35688/0 (socket says 172.21.15.37:35688) 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 -- 172.21.15.37:0/4089138965 learned_addr learned my addr 172.21.15.37:0/4089138965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f15137fe640 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cdf40 0x55c85c7d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1513fff640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 0x7f14fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1510ff9640 1 -- 172.21.15.37:0/208456683 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f150c0d08c0 con 0x7f151419cf70 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 -- 172.21.15.37:0/4089138965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 msgr2=0x55c85c7cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 -- 172.21.15.37:0/4089138965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cdf40 msgr2=0x55c85c7d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cdf40 0x55c85c7d82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 -- 172.21.15.37:0/4089138965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85c445210 con 0x55c85c7d8800 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1512ffd640 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15080058c0 tx=0x7f1508005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39a0b263e41c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f1513fff640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 0x7f14fc0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f150c09c5e0 tx=0x7f150c087040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f14f9ffb640 1 -- 172.21.15.37:0/4089138965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508004030 con 0x55c85c7d8800 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f14f9ffb640 1 -- 172.21.15.37:0/4089138965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1508005020 con 0x55c85c7d8800 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.581+0000 7f14f9ffb640 1 -- 172.21.15.37:0/4089138965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508004b20 con 0x55c85c7d8800 2024-04-25T21:12:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 -- 172.21.15.37:0/4089138965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 msgr2=0x55c85c7dabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15080058c0 tx=0x7f1508005850 comp rx=0 tx=0).stop 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 -- 172.21.15.37:0/4089138965 shutdown_connections 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7dabf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cdf40 0x55c85c7d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 --2- 172.21.15.37:0/4089138965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 -- 172.21.15.37:0/4089138965 >> 172.21.15.37:0/4089138965 conn(0x55c85c7c9b30 msgr2=0x55c85c7c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 -- 172.21.15.37:0/4089138965 shutdown_connections 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.582+0000 7f151c7e9740 1 -- 172.21.15.37:0/4089138965 wait complete. 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 -- start start 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 Processor -- start 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 0x55c85c7d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c85c7db430 con 0x55c85c7d8800 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c85c6cd0b0 con 0x55c85c7cd620 2024-04-25T21:12:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c85c7db5b0 con 0x55c85c7cdf40 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f151c7e9740 10 client.0 _ll_register_callbacks cb 0x55c85c6bad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f1512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f1512ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35704/0 (socket says 172.21.15.37:35704) 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f1512ffd640 1 -- 172.21.15.37:0/1798369717 learned_addr learned my addr 172.21.15.37:0/1798369717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f15137fe640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 0x55c85c7d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.583+0000 7f1512ffd640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 -- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 msgr2=0x55c85c7d5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 0x55c85c7d5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 -- 172.21.15.37:0/1798369717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 msgr2=0x55c85c7d81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7d81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 -- 172.21.15.37:0/1798369717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85c511620 con 0x55c85c7d8800 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f15137fe640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 0x55c85c7d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f1512ffd640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1508005690 tx=0x7f1508009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136565b7fe5ce4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f14f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508004320 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f1508019040 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1508005020 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1508033050 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f151c7e9740 10 client.15879 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c85c7e73c0 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.584+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1508005020 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f150800fac0 con 0x55c85c7d8800 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 10 client.15879 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 20 client.15879 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c7e87c0 0x55c85c7eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 4 client.15879 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c85c740390 con 0x55c85c7e87c0 2024-04-25T21:12:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.585+0000 7f15137fe640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c7e87c0 0x55c85c7eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f14f2ffd640 10 client.15879 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f15137fe640 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c7e87c0 0x55c85c7eabb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6fbf0c0511637c server_cookie=83ad159fa21dc1c in_seq=0 out_seq=0 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f14f2ffd640 1 -- 172.21.15.37:0/1798369717 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c85c740390 con 0x55c85c7e87c0 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f14f2ffd640 10 client.15879 handle_command_reply: tid=0 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f151c7e9740 1 client.15879 shutdown 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.586+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 msgr2=0x55c85c7d5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1508005690 tx=0x7f1508009d10 comp rx=0 tx=0).stop 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 shutdown_connections 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c7e87c0 0x55c85c7eabb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7d8800 0x55c85c7d5a30 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cdf40 0x55c85c7d5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f14f2ffd640 0 client.15879 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 --2- 172.21.15.37:0/1798369717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7d81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 >> 172.21.15.37:0/1798369717 conn(0x55c85c7c9b30 msgr2=0x55c85c7cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 shutdown_connections 2024-04-25T21:12:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 1 -- 172.21.15.37:0/1798369717 wait complete. 2024-04-25T21:12:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.587+0000 7f151c7e9740 20 client.15879 trim_cache size 0 max 16384 2024-04-25T21:12:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 Processor -- start 2024-04-25T21:12:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 -- start start 2024-04-25T21:12:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7d8800 0x55c85c7d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15141ada70 con 0x55c85c7cdf40 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f15141af5b0 con 0x55c85c7d8800 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f151c7e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15141ae140 con 0x55c85c7cd620 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35712/0 (socket says 172.21.15.37:35712) 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 -- 172.21.15.37:0/520601391 learned_addr learned my addr 172.21.15.37:0/520601391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 -- 172.21.15.37:0/520601391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 msgr2=0x55c85c7d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 -- 172.21.15.37:0/520601391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7d8800 msgr2=0x55c85c7d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7d8800 0x55c85c7d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 -- 172.21.15.37:0/520601391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85c7d5180 con 0x55c85c7cdf40 2024-04-25T21:12:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.602+0000 7f15137fe640 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1504031440 tx=0x7f1504009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3398fbba35f908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f14f97fa640 1 -- 172.21.15.37:0/520601391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150404a070 con 0x55c85c7cdf40 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f14f97fa640 1 -- 172.21.15.37:0/520601391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15040054f0 con 0x55c85c7cdf40 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f14f97fa640 1 -- 172.21.15.37:0/520601391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504035040 con 0x55c85c7cdf40 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 -- 172.21.15.37:0/520601391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 msgr2=0x55c85c7d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1504031440 tx=0x7f1504009640 comp rx=0 tx=0).stop 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 -- 172.21.15.37:0/520601391 shutdown_connections 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7d8800 0x55c85c7d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7d08c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 --2- 172.21.15.37:0/520601391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7cd620 0x55c85c7d4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 -- 172.21.15.37:0/520601391 >> 172.21.15.37:0/520601391 conn(0x55c85c7c9b30 msgr2=0x55c85c7d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 -- 172.21.15.37:0/520601391 shutdown_connections 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.603+0000 7f151c7e9740 1 -- 172.21.15.37:0/520601391 wait complete. 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 -- start start 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 Processor -- start 2024-04-25T21:12:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7d8800 0x55c85c7db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15141aeee0 con 0x55c85c7cdf40 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f15141ae810 con 0x55c85c7cd620 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15141aba70 con 0x55c85c7d8800 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f151c7e9740 10 client.0 _ll_register_callbacks cb 0x7f1508011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f15137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f15137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35724/0 (socket says 172.21.15.37:35724) 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.604+0000 7f15137fe640 1 -- 172.21.15.37:0/7969014 learned_addr learned my addr 172.21.15.37:0/7969014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f1512ffd640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7d8800 0x55c85c7db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f1512ffd640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 -- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7d8800 msgr2=0x55c85c7db9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7d8800 0x55c85c7db9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 -- 172.21.15.37:0/7969014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 msgr2=0x55c85c7e0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 -- 172.21.15.37:0/7969014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85c7dc110 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f15137fe640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1504009230 tx=0x7f1504005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f39596b5f89870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f14f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150404a040 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f15040035f0 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15040054f0 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1504035040 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f151c7e9740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c85c84e5d0 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.605+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15040054f0 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f150404d070 con 0x55c85c7cdf40 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 20 client.15885 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c84ed40 0x55c85c851160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:40.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 4 client.15885 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:12:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c85c73a200 con 0x55c85c84ed40 2024-04-25T21:12:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f1512ffd640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c84ed40 0x55c85c851160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f14f2ffd640 10 client.15885 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.606+0000 7f1512ffd640 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c84ed40 0x55c85c851160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d690d546242bd4 server_cookie=9a3af2f8bf6210b9 in_seq=0 out_seq=0 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f14f2ffd640 1 -- 172.21.15.37:0/7969014 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c85c73a200 con 0x55c85c84ed40 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f14f2ffd640 10 client.15885 handle_command_reply: tid=0 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 client.15885 shutdown 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 msgr2=0x55c85c7e0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1504009230 tx=0x7f1504005fd0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 shutdown_connections 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c85c84ed40 0x55c85c851160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c85c7d8800 0x55c85c7db9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c85c7cdf40 0x55c85c7e0a50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f14f2ffd640 0 client.15885 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 --2- 172.21.15.37:0/7969014 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c85c7cd620 0x55c85c7e0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 >> 172.21.15.37:0/7969014 conn(0x55c85c7c9b30 msgr2=0x55c85c7d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 shutdown_connections 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.620+0000 7f151c7e9740 1 -- 172.21.15.37:0/7969014 wait complete. 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f151c7e9740 20 client.15885 trim_cache size 0 max 16384 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:12:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 msgr2=0x7f14fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 0x7f14fc0a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f150c09c5e0 tx=0x7f150c087040 comp rx=0 tx=0).stop 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 msgr2=0x7f151419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f150c00eaa0 tx=0x7f150c00ef60 comp rx=0 tx=0).stop 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 shutdown_connections 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f151419cf70 0x7f151419d420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14fc0a3dc0 0x7f14fc0a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15141061b0 0x7f1514198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 --2- 172.21.15.37:0/208456683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15141010e0 0x7f15141986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 >> 172.21.15.37:0/208456683 conn(0x7f15140771d0 msgr2=0x7f151406aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 shutdown_connections 2024-04-25T21:12:40.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:40.621+0000 7f1519acf640 1 -- 172.21.15.37:0/208456683 wait complete. 2024-04-25T21:12:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:41 smithi098 ceph-mon[32569]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T21:12:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:41 smithi037 ceph-mon[29195]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T21:12:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:41 smithi177 ceph-mon[33043]: pgmap v129: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-04-25T21:12:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:43 smithi098 ceph-mon[32569]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T21:12:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:43 smithi037 ceph-mon[29195]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T21:12:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:43 smithi177 ceph-mon[33043]: pgmap v130: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 9 op/s 2024-04-25T21:12:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:45 smithi098 ceph-mon[32569]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:12:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:45 smithi037 ceph-mon[29195]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:12:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:45 smithi177 ceph-mon[33043]: pgmap v131: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:12:47.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:47 smithi098 ceph-mon[32569]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:47 smithi037 ceph-mon[29195]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:47 smithi177 ceph-mon[33043]: pgmap v132: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:49 smithi098 ceph-mon[32569]: pgmap v133: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:49 smithi037 ceph-mon[29195]: pgmap v133: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:49 smithi177 ceph-mon[33043]: pgmap v133: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 6.8 KiB/s wr, 11 op/s 2024-04-25T21:12:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:51 smithi098 ceph-mon[32569]: pgmap v134: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T21:12:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:51 smithi037 ceph-mon[29195]: pgmap v134: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T21:12:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:51 smithi177 ceph-mon[33043]: pgmap v134: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.8 KiB/s wr, 15 op/s 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 -- 172.21.15.37:0/412340095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 msgr2=0x7f3668110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 --2- 172.21.15.37:0/412340095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f3668110b00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3664009930 tx=0x7f366402a420 comp rx=0 tx=0).stop 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 -- 172.21.15.37:0/412340095 shutdown_connections 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 --2- 172.21.15.37:0/412340095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 0x7f36681175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 --2- 172.21.15.37:0/412340095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 0x7f366810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 --2- 172.21.15.37:0/412340095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f3668110b00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 -- 172.21.15.37:0/412340095 >> 172.21.15.37:0/412340095 conn(0x7f366806f090 msgr2=0x7f3668108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.726+0000 7f366f462640 1 -- 172.21.15.37:0/412340095 shutdown_connections 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 -- 172.21.15.37:0/412340095 wait complete. 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 Processor -- start 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 -- start start 2024-04-25T21:12:53.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 0x7f36681a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 0x7f36681a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f366811a9e0 con 0x7f3668110720 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f366811a860 con 0x7f366810a760 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366f462640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f366811ab60 con 0x7f366810b120 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.727+0000 7f366d9ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53038/0 (socket says 172.21.15.37:53038) 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 -- 172.21.15.37:0/1627644647 learned_addr learned my addr 172.21.15.37:0/1627644647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 0x7f36681a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 0x7f36681a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 -- 172.21.15.37:0/1627644647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 msgr2=0x7f36681a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 0x7f36681a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 -- 172.21.15.37:0/1627644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 msgr2=0x7f36681a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 0x7f36681a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 -- 172.21.15.37:0/1627644647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36681a5ea0 con 0x7f3668110720 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366d9ff640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f366000eaa0 tx=0x7f366000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f365e7fc640 1 -- 172.21.15.37:0/1627644647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000cdb0 con 0x7f3668110720 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f365e7fc640 1 -- 172.21.15.37:0/1627644647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3660005020 con 0x7f3668110720 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36681a6190 con 0x7f3668110720 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f365e7fc640 1 -- 172.21.15.37:0/1627644647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600116d0 con 0x7f3668110720 2024-04-25T21:12:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.728+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681ada20 con 0x7f3668110720 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f365e7fc640 1 -- 172.21.15.37:0/1627644647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f36600051d0 con 0x7f3668110720 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 Processor -- start 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 -- start start 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f365e7fc640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 0x7f36540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556d2b50b970 con 0x556d2b5b5c10 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556d2b4a84a0 con 0x556d2b5a6e40 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.731+0000 7f3672170740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556d2b507930 con 0x556d2b5a6520 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d9ff640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 0x7f36540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:50648/0 (socket says 172.21.15.37:50648) 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53040/0 (socket says 172.21.15.37:53040) 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f365e7fc640 1 -- 172.21.15.37:0/1627644647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f36600d0940 con 0x7f3668110720 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366c9fd640 1 -- 172.21.15.37:0/2320640766 learned_addr learned my addr 172.21.15.37:0/2320640766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366c9fd640 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 -- 172.21.15.37:0/2320640766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 msgr2=0x556d2b5a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 -- 172.21.15.37:0/2320640766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 msgr2=0x556d2b5b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 -- 172.21.15.37:0/2320640766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2b21e210 con 0x556d2b5a6e40 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366c9fd640 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d9ff640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 0x7f36540a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f36600ce170 tx=0x7f3660091da0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f366d1fe640 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f3664032ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83244b847b24f3dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f36537fe640 1 -- 172.21.15.37:0/2320640766 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664035030 con 0x556d2b5a6e40 2024-04-25T21:12:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f36537fe640 1 -- 172.21.15.37:0/2320640766 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3664032e50 con 0x556d2b5a6e40 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.732+0000 7f36537fe640 1 -- 172.21.15.37:0/2320640766 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664005040 con 0x556d2b5a6e40 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 -- 172.21.15.37:0/2320640766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 msgr2=0x556d2b5b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f3664032ba0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 -- 172.21.15.37:0/2320640766 shutdown_connections 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b56d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 --2- 172.21.15.37:0/2320640766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 -- 172.21.15.37:0/2320640766 >> 172.21.15.37:0/2320640766 conn(0x556d2b5a2b30 msgr2=0x556d2b5a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 -- 172.21.15.37:0/2320640766 shutdown_connections 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.733+0000 7f3672170740 1 -- 172.21.15.37:0/2320640766 wait complete. 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 -- start start 2024-04-25T21:12:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 Processor -- start 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6520 0x556d2b5b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6e40 0x556d2b5b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556d2b5b87e0 con 0x556d2b5b5c10 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556d2b4a60b0 con 0x556d2b5a6520 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556d2b5b8960 con 0x556d2b5a6e40 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f3672170740 10 client.0 _ll_register_callbacks cb 0x556d2b493d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53044/0 (socket says 172.21.15.37:53044) 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f366c9fd640 1 -- 172.21.15.37:0/292713815 learned_addr learned my addr 172.21.15.37:0/292713815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f366d1fe640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6e40 0x556d2b5b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.734+0000 7f366c9fd640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6520 0x556d2b5b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 -- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6e40 msgr2=0x556d2b5b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6e40 0x556d2b5b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 -- 172.21.15.37:0/292713815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6520 msgr2=0x556d2b5b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6520 0x556d2b5b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 -- 172.21.15.37:0/292713815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b2ea620 con 0x556d2b5b5c10 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f366c9fd640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3658012d00 tx=0x7f3658004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee66a22f6eacf6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658012050 con 0x556d2b5b5c10 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f36580121f0 con 0x556d2b5b5c10 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3664032e50 con 0x556d2b5b5c10 2024-04-25T21:12:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f36580123f0 con 0x556d2b5b5c10 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3672170740 10 client.15897 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d2b5c4760 con 0x556d2b5b5c10 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.735+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3664032e50 con 0x556d2b5b5c10 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36580047a0 con 0x556d2b5b5c10 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 10 client.15897 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 20 client.15897 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5c5bd0 0x556d2b5c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 4 client.15897 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2b519390 con 0x556d2b5c5bd0 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f366d1fe640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5c5bd0 0x556d2b5c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f366d1fe640 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5c5bd0 0x556d2b5c7fc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e93c89b7661fda7 server_cookie=2e42a4d1799b8bcf in_seq=0 out_seq=0 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.736+0000 7f3650ff9640 10 client.15897 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3650ff9640 1 -- 172.21.15.37:0/292713815 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d2b519390 con 0x556d2b5c5bd0 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3650ff9640 10 client.15897 handle_command_reply: tid=0 2024-04-25T21:12:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 client.15897 shutdown 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 msgr2=0x556d2b5b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3658012d00 tx=0x7f3658004730 comp rx=0 tx=0).stop 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 shutdown_connections 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5c5bd0 0x556d2b5c7fc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5b5c10 0x556d2b5b2d10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6e40 0x556d2b5b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3650ff9640 0 client.15897 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 --2- 172.21.15.37:0/292713815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6520 0x556d2b5b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.737+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 >> 172.21.15.37:0/292713815 conn(0x556d2b5a2b30 msgr2=0x556d2b5a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.738+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 shutdown_connections 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.738+0000 7f3672170740 1 -- 172.21.15.37:0/292713815 wait complete. 2024-04-25T21:12:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.738+0000 7f3672170740 20 client.15897 trim_cache size 0 max 16384 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 Processor -- start 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 -- start start 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b4070 con 0x556d2b5a2b30 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b5bb0 con 0x556d2b5a6e40 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f3672170740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b4740 con 0x556d2b5a6520 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53050/0 (socket says 172.21.15.37:53050) 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 -- 172.21.15.37:0/1489418330 learned_addr learned my addr 172.21.15.37:0/1489418330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 -- 172.21.15.37:0/1489418330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 msgr2=0x556d2b5add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 -- 172.21.15.37:0/1489418330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 msgr2=0x556d2b5b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 -- 172.21.15.37:0/1489418330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2b5c85f0 con 0x556d2b5a2b30 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.753+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f36580059f0 tx=0x7f3658012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec958cc0ea4c242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3652ffd640 1 -- 172.21.15.37:0/1489418330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658009910 con 0x556d2b5a2b30 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3652ffd640 1 -- 172.21.15.37:0/1489418330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365802e070 con 0x556d2b5a2b30 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3652ffd640 1 -- 172.21.15.37:0/1489418330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658004ff0 con 0x556d2b5a2b30 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 -- 172.21.15.37:0/1489418330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 msgr2=0x556d2b5c16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f36580059f0 tx=0x7f3658012a30 comp rx=0 tx=0).stop 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 -- 172.21.15.37:0/1489418330 shutdown_connections 2024-04-25T21:12:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a6520 0x556d2b5add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 --2- 172.21.15.37:0/1489418330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a2b30 0x556d2b5c16a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 -- 172.21.15.37:0/1489418330 >> 172.21.15.37:0/1489418330 conn(0x556d2b5b2e70 msgr2=0x556d2b5ad670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 -- 172.21.15.37:0/1489418330 shutdown_connections 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.754+0000 7f3672170740 1 -- 172.21.15.37:0/1489418330 wait complete. 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 -- start start 2024-04-25T21:12:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 Processor -- start 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a2b30 0x556d2b5bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5af560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b54e0 con 0x556d2b5a6520 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b4e10 con 0x556d2b5a6e40 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f36681b2050 con 0x556d2b5a2b30 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f3672170740 10 client.0 _ll_register_callbacks cb 0x7f3658012b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f366d1fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f366d1fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53058/0 (socket says 172.21.15.37:53058) 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f366d1fe640 1 -- 172.21.15.37:0/1693845973 learned_addr learned my addr 172.21.15.37:0/1693845973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.755+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a2b30 0x556d2b5bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 -- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a2b30 msgr2=0x556d2b5bdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a2b30 0x556d2b5bdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 -- 172.21.15.37:0/1693845973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 msgr2=0x556d2b5af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:12:53.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 -- 172.21.15.37:0/1693845973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b5afb20 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f366d1fe640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3664009930 tx=0x7f3664005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d490b04cfc0dc7c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664035040 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f3664005240 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365802e070 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f366405f050 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3672170740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556d2b62bed0 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.756+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365802e070 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f36640035f0 con 0x556d2b5a6520 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:12:53.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 20 client.15903 populate_metadata read hostname 'smithi037' 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5b5c10 0x556d2b62e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 4 client.15903 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d2b513200 con 0x556d2b5b5c10 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5b5c10 0x556d2b62e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f366c9fd640 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5b5c10 0x556d2b62e210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f43c2f4d65a6d9a server_cookie=a2a901c700c60c90 in_seq=0 out_seq=0 2024-04-25T21:12:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.757+0000 7f3650ff9640 10 client.15903 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.768+0000 7f3650ff9640 1 -- 172.21.15.37:0/1693845973 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d2b513200 con 0x556d2b5b5c10 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.768+0000 7f3650ff9640 10 client.15903 handle_command_reply: tid=0 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.768+0000 7f3672170740 1 client.15903 shutdown 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 msgr2=0x556d2b5af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3664009930 tx=0x7f3664005a20 comp rx=0 tx=0).stop 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 shutdown_connections 2024-04-25T21:12:53.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556d2b5b5c10 0x556d2b62e210 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556d2b5a6e40 0x556d2b5af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556d2b5a6520 0x556d2b5af020 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 --2- 172.21.15.37:0/1693845973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556d2b5a2b30 0x556d2b5bdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3650ff9640 0 client.15903 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 >> 172.21.15.37:0/1693845973 conn(0x556d2b5b2e70 msgr2=0x556d2b5a1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 shutdown_connections 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 1 -- 172.21.15.37:0/1693845973 wait complete. 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.769+0000 7f3672170740 20 client.15903 trim_cache size 0 max 16384 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:12:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:12:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 msgr2=0x7f36540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 0x7f36540a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f36600ce170 tx=0x7f3660091da0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 msgr2=0x7f36681a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f366000eaa0 tx=0x7f366000ef60 comp rx=0 tx=0).stop 2024-04-25T21:12:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 shutdown_connections 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3668110720 0x7f36681a5770 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f36540a3dc0 0x7f36540a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366810b120 0x7f36681a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 --2- 172.21.15.37:0/1627644647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f366810a760 0x7f36681a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 >> 172.21.15.37:0/1627644647 conn(0x7f366806f090 msgr2=0x7f366806f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 shutdown_connections 2024-04-25T21:12:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:12:53.770+0000 7f366f462640 1 -- 172.21.15.37:0/1627644647 wait complete. 2024-04-25T21:12:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:53 smithi098 ceph-mon[32569]: pgmap v135: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-04-25T21:12:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:53 smithi037 ceph-mon[29195]: pgmap v135: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-04-25T21:12:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:53 smithi177 ceph-mon[33043]: pgmap v135: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 11 op/s 2024-04-25T21:12:55.465 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:12:55.733 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:12:55.855 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:55 smithi037 ceph-mon[29195]: pgmap v136: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2024-04-25T21:12:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:55 smithi098 ceph-mon[32569]: pgmap v136: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2024-04-25T21:12:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:55 smithi177 ceph-mon[33043]: pgmap v136: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2024-04-25T21:12:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/657919871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74101030 msgr2=0x7f2a74101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/657919871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74101030 0x7f2a74101490 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009990 tx=0x7f2a68009ca0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/657919871 shutdown_connections 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/657919871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a741019d0 0x7f2a7410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/657919871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a74108ad0 0x7f2a74108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/657919871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74101030 0x7f2a74101490 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/657919871 >> 172.21.15.37:0/657919871 conn(0x7f2a74077fb0 msgr2=0x7f2a740ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/657919871 shutdown_connections 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.597+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/657919871 wait complete. 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 Processor -- start 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 -- start start 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a74101030 0x7f2a74102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741019d0 0x7f2a74102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74112be0 con 0x7f2a74108ad0 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74112a60 con 0x7f2a741019d0 2024-04-25T21:12:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a7a7c4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74112d60 con 0x7f2a74101030 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a78d61700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a78d61700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53078/0 (socket says 172.21.15.37:53078) 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a78d61700 1 -- 172.21.15.37:0/3375495902 learned_addr learned my addr 172.21.15.37:0/3375495902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.598+0000 7f2a73fff700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a74101030 0x7f2a74102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a737fe700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741019d0 0x7f2a74102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 -- 172.21.15.37:0/3375495902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a74101030 msgr2=0x7f2a74102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a74101030 0x7f2a74102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 -- 172.21.15.37:0/3375495902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741019d0 msgr2=0x7f2a74102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741019d0 0x7f2a74102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 -- 172.21.15.37:0/3375495902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a741a7510 con 0x7f2a74108ad0 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a78d61700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007d70 tx=0x7f2a6c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:57.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c016020 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a741a7800 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6c00ad40 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c0115e0 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.599+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a741a7d40 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.600+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a74069730 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.601+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a6c019020 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.602+0000 7f2a717fa700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 0x7f2a5c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.602+0000 7f2a73fff700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 0x7f2a5c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.602+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2a6c0d0780 con 0x7f2a74108ad0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.602+0000 7f2a73fff700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 0x7f2a5c0a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a6400af20 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:12:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.607+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a6c020020 con 0x7f2a74108ad0 2024-04-25T21:12:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.864+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7402df80 con 0x7f2a74108ad0 2024-04-25T21:12:57.863 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:57 smithi037 ceph-mon[29195]: pgmap v137: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:12:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.865+0000 7f2a717fa700 1 -- 172.21.15.37:0/3375495902 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f2a6c09cf00 con 0x7f2a74108ad0 2024-04-25T21:12:57.864 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:12:57.864 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:12:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 msgr2=0x7f2a5c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 0x7f2a5c0a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a6400af20 comp rx=0 tx=0).stop 2024-04-25T21:12:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 msgr2=0x7f2a74102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:12:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c007d70 tx=0x7f2a6c00a340 comp rx=0 tx=0).stop 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 shutdown_connections 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2a5c0a3db0 0x7f2a5c0a6270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a74108ad0 0x7f2a74102c90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a74101030 0x7f2a74102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 --2- 172.21.15.37:0/3375495902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a741019d0 0x7f2a74102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 >> 172.21.15.37:0/3375495902 conn(0x7f2a74077fb0 msgr2=0x7f2a740ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 shutdown_connections 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:12:57.867+0000 7f2a7a7c4700 1 -- 172.21.15.37:0/3375495902 wait complete. 2024-04-25T21:12:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:12:57.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:57 smithi098 ceph-mon[32569]: pgmap v137: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:12:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:57 smithi177 ceph-mon[33043]: pgmap v137: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:12:58.637 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id cd372993 --release --await' 2024-04-25T21:12:58.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id cd372993 --release --await 2024-04-25T21:12:58.894 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:12:58.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:58 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3375495902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:58 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3375495902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:59.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:58 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3375495902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:12:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:12:59 smithi037 ceph-mon[29195]: pgmap v138: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:13:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:12:59 smithi177 ceph-mon[33043]: pgmap v138: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:13:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:12:59 smithi098 ceph-mon[32569]: pgmap v138: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 -- 172.21.15.37:0/2917048540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 msgr2=0x7f5098108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 --2- 172.21.15.37:0/2917048540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f5098108e30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f509000b0a0 tx=0x7f509000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 -- 172.21.15.37:0/2917048540 shutdown_connections 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 --2- 172.21.15.37:0/2917048540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098103450 0x7f509810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 --2- 172.21.15.37:0/2917048540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f5098108e30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 --2- 172.21.15.37:0/2917048540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098102a90 0x7f5098102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 -- 172.21.15.37:0/2917048540 >> 172.21.15.37:0/2917048540 conn(0x7f5098079ba0 msgr2=0x7f5098079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 -- 172.21.15.37:0/2917048540 shutdown_connections 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.771+0000 7f50a040f700 1 -- 172.21.15.37:0/2917048540 wait complete. 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 Processor -- start 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 -- start start 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098102a90 0x7f5098074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 0x7f5098075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5098112c10 con 0x7f5098108a50 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5098112a90 con 0x7f5098102a90 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f50a040f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5098112d90 con 0x7f5098103450 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51300/0 (socket says 172.21.15.37:51300) 2024-04-25T21:13:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f509e9ac700 1 -- 172.21.15.37:0/2561007389 learned_addr learned my addr 172.21.15.37:0/2561007389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f509e1ab700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098102a90 0x7f5098074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.772+0000 7f509d9aa700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 0x7f5098075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 -- 172.21.15.37:0/2561007389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 msgr2=0x7f5098075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 0x7f5098075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 -- 172.21.15.37:0/2561007389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098102a90 msgr2=0x7f5098074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098102a90 0x7f5098074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 -- 172.21.15.37:0/2561007389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50981a7690 con 0x7f5098108a50 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509d9aa700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 0x7f5098075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f509e9ac700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f508800c970 tx=0x7f508800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f50977fe700 1 -- 172.21.15.37:0/2561007389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508800ce60 con 0x7f5098108a50 2024-04-25T21:13:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f50977fe700 1 -- 172.21.15.37:0/2561007389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5088004830 con 0x7f5098108a50 2024-04-25T21:13:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50981a7980 con 0x7f5098108a50 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f50977fe700 1 -- 172.21.15.37:0/2561007389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880054c0 con 0x7f5098108a50 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.773+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a7ec0 con 0x7f5098108a50 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.776+0000 7f50977fe700 1 -- 172.21.15.37:0/2561007389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50880049e0 con 0x7f5098108a50 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.776+0000 7f50977fe700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 0x7f50800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.776+0000 7f509e1ab700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 0x7f50800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.776+0000 7f50977fe700 1 -- 172.21.15.37:0/2561007389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f50880d05e0 con 0x7f5098108a50 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 Processor -- start 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 -- start start 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f509e1ab700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 0x7f50800a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fd0 tx=0x7f509000b7b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dacdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1db7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559bd1cffe30 con 0x559bd1db81e0 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559bd1ca6a60 con 0x559bd1dad310 2024-04-25T21:13:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f50aac98b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559bd1cfbdf0 con 0x559bd1dac9f0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.777+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51306/0 (socket says 172.21.15.37:51306) 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 -- 172.21.15.37:0/1535267874 learned_addr learned my addr 172.21.15.37:0/1535267874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e1ab700 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dacdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509d9aa700 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1db7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 -- 172.21.15.37:0/1535267874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 msgr2=0x559bd1dacdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dacdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 -- 172.21.15.37:0/1535267874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 msgr2=0x559bd1db7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1db7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 -- 172.21.15.37:0/1535267874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd1c4d9d0 con 0x559bd1db81e0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f509e9ac700 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f508809c540 tx=0x7f508809ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2793b6d8077f098 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f5094d70700 1 -- 172.21.15.37:0/1535267874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508801d040 con 0x559bd1db81e0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f5094d70700 1 -- 172.21.15.37:0/1535267874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50880a5220 con 0x559bd1db81e0 2024-04-25T21:13:00.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.778+0000 7f5094d70700 1 -- 172.21.15.37:0/1535267874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508809c810 con 0x559bd1db81e0 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- 172.21.15.37:0/1535267874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 msgr2=0x559bd1dba5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f508809c540 tx=0x7f508809ccb0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- 172.21.15.37:0/1535267874 shutdown_connections 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dba5d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dacdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 --2- 172.21.15.37:0/1535267874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1db7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- 172.21.15.37:0/1535267874 >> 172.21.15.37:0/1535267874 conn(0x559bd1cda790 msgr2=0x559bd1da8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- 172.21.15.37:0/1535267874 shutdown_connections 2024-04-25T21:13:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- 172.21.15.37:0/1535267874 wait complete. 2024-04-25T21:13:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.779+0000 7f50aac98b80 1 -- start start 2024-04-25T21:13:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 Processor -- start 2024-04-25T21:13:00.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1db7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1db81e0 0x559bd1db5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559bd1dbabd0 con 0x559bd1dad310 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559bd1c9ecd0 con 0x559bd1dac9f0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559bd1dbad50 con 0x559bd1db81e0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f50aac98b80 10 client.0 _ll_register_callbacks cb 0x559bd1c98a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51310/0 (socket says 172.21.15.37:51310) 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509e1ab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1db7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 -- 172.21.15.37:0/962746742 learned_addr learned my addr 172.21.15.37:0/962746742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509e9ac700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1db81e0 0x559bd1db5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 -- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1db81e0 msgr2=0x559bd1db5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1db81e0 0x559bd1db5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 -- 172.21.15.37:0/962746742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 msgr2=0x559bd1db7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1db7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509d9aa700 1 -- 172.21.15.37:0/962746742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd1cb7370 con 0x559bd1dad310 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.780+0000 7f509e1ab700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1db7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f509d9aa700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f508c010840 tx=0x7f508c010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4038410d598dbd3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f507dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c011970 con 0x559bd1dad310 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f508c011b10 con 0x559bd1dad310 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50880a5220 con 0x559bd1dad310 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f508c011d50 con 0x559bd1dad310 2024-04-25T21:13:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f50aac98b80 10 client.15921 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bd1dc5470 con 0x559bd1dad310 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.781+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50880a5220 con 0x559bd1dad310 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f508c00e460 con 0x559bd1dad310 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 10 client.15921 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 20 client.15921 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1dc7ab0 0x559bd1dc9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 4 client.15921 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd1d0d620 con 0x559bd1dc7ab0 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f509e1ab700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1dc7ab0 0x559bd1dc9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f509e1ab700 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1dc7ab0 0x559bd1dc9ea0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18ead3095e148f6 server_cookie=d17c13ad0bf29e60 in_seq=0 out_seq=0 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.782+0000 7f507dffb700 10 client.15921 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.783+0000 7f507dffb700 1 -- 172.21.15.37:0/962746742 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559bd1d0d620 con 0x559bd1dc7ab0 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.783+0000 7f507dffb700 10 client.15921 handle_command_reply: tid=0 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.783+0000 7f50aac98b80 1 client.15921 shutdown 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.783+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 msgr2=0x559bd1db4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.783+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f508c010840 tx=0x7f508c010b50 comp rx=0 tx=0).stop 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 shutdown_connections 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1dc7ab0 0x559bd1dc9ea0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1db81e0 0x559bd1db5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1db7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f507dffb700 0 client.15921 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 --2- 172.21.15.37:0/962746742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1dad310 0x559bd1db4c30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 >> 172.21.15.37:0/962746742 conn(0x559bd1cda790 msgr2=0x559bd1da8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 shutdown_connections 2024-04-25T21:13:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 1 -- 172.21.15.37:0/962746742 wait complete. 2024-04-25T21:13:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.784+0000 7f50aac98b80 20 client.15921 trim_cache size 0 max 16384 2024-04-25T21:13:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 Processor -- start 2024-04-25T21:13:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 -- start start 2024-04-25T21:13:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1dcf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 0x559bd1dcfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50981af5d0 con 0x559bd1db81e0 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50981b1110 con 0x559bd1dac9f0 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f50aac98b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50981afca0 con 0x559bd1dad310 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.803+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51314/0 (socket says 172.21.15.37:51314) 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 -- 172.21.15.37:0/820472050 learned_addr learned my addr 172.21.15.37:0/820472050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509d9aa700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 0x559bd1dcfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 -- 172.21.15.37:0/820472050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 msgr2=0x559bd1dcfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 0x559bd1dcfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e1ab700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1dcf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 -- 172.21.15.37:0/820472050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 msgr2=0x559bd1dcf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1dcf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 -- 172.21.15.37:0/820472050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd1daa370 con 0x559bd1db81e0 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509d9aa700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 0x559bd1dcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f509e9ac700 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f508809a860 tx=0x7f508800a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a565cbfce99461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f507ffff700 1 -- 172.21.15.37:0/820472050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508801d040 con 0x559bd1db81e0 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f507ffff700 1 -- 172.21.15.37:0/820472050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5088005660 con 0x559bd1db81e0 2024-04-25T21:13:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.804+0000 7f507ffff700 1 -- 172.21.15.37:0/820472050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508809c810 con 0x559bd1db81e0 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- 172.21.15.37:0/820472050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 msgr2=0x559bd1dbff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f508809a860 tx=0x7f508800a590 comp rx=0 tx=0).stop 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- 172.21.15.37:0/820472050 shutdown_connections 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbff90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dac9f0 0x559bd1dcf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 --2- 172.21.15.37:0/820472050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dad310 0x559bd1dcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- 172.21.15.37:0/820472050 >> 172.21.15.37:0/820472050 conn(0x559bd1cda790 msgr2=0x559bd1db3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- 172.21.15.37:0/820472050 shutdown_connections 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- 172.21.15.37:0/820472050 wait complete. 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 -- start start 2024-04-25T21:13:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.805+0000 7f50aac98b80 1 Processor -- start 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1dbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50981b0a40 con 0x559bd1db81e0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50981b0370 con 0x559bd1dad310 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ad570 con 0x559bd1dac9f0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 10 client.0 _ll_register_callbacks cb 0x559bd1c98a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51316/0 (socket says 172.21.15.37:51316) 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 -- 172.21.15.37:0/549317206 learned_addr learned my addr 172.21.15.37:0/549317206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e1ab700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509d9aa700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1dbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 -- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 msgr2=0x559bd1dc0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dc0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 -- 172.21.15.37:0/549317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 msgr2=0x559bd1dbcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1dbcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 -- 172.21.15.37:0/549317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd1dbdaf0 con 0x559bd1db81e0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509d9aa700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1dbcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f509e9ac700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f508809c4f0 tx=0x7f50880a0a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb05c2660240d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f507dffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508801d070 con 0x559bd1db81e0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f508809ca10 con 0x559bd1db81e0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.806+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5088005660 con 0x559bd1db81e0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5088082050 con 0x559bd1db81e0 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 10 client.15927 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:00.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559bd1e2ee70 con 0x559bd1db81e0 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5088005660 con 0x559bd1db81e0 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f50880d00f0 con 0x559bd1db81e0 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 10 client.15927 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 20 client.15927 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1e2f040 0x559bd1e31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 4 client.15927 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cd372993", "release": true, "await": true}] 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cd372993", "release": true, "await": true}) -- 0x559bd1d076c0 con 0x559bd1e2f040 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.807+0000 7f509e1ab700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1e2f040 0x559bd1e31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.808+0000 7f509e1ab700 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1e2f040 0x559bd1e31430 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7c5128ed77901f8 server_cookie=21eff2ed98ac6743 in_seq=0 out_seq=0 2024-04-25T21:13:00.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.808+0000 7f507dffb700 10 client.15927 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f507dffb700 1 -- 172.21.15.37:0/549317206 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559bd1d076c0 con 0x559bd1e2f040 2024-04-25T21:13:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f507dffb700 10 client.15927 handle_command_reply: tid=0 2024-04-25T21:13:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 client.15927 shutdown 2024-04-25T21:13:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 msgr2=0x559bd1dbd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f508809c4f0 tx=0x7f50880a0a20 comp rx=0 tx=0).stop 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 shutdown_connections 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559bd1e2f040 0x559bd1e31430 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f507dffb700 0 client.15927 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd1db81e0 0x559bd1dbd4b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559bd1dac9f0 0x559bd1dc0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 --2- 172.21.15.37:0/549317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559bd1dad310 0x559bd1dbcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.811+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 >> 172.21.15.37:0/549317206 conn(0x559bd1cda790 msgr2=0x559bd1db6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 shutdown_connections 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50aac98b80 1 -- 172.21.15.37:0/549317206 wait complete. 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50aac98b80 20 client.15927 trim_cache size 0 max 16384 2024-04-25T21:13:00.813 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "cd372993": { 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:13:00.814 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.8, 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:00.815 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 msgr2=0x7f50800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 0x7f50800a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5090009fd0 tx=0x7f509000b7b0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 msgr2=0x7f50980757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f508800c970 tx=0x7f508800cc80 comp rx=0 tx=0).stop 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 shutdown_connections 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50800a3db0 0x7f50800a6270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5098108a50 0x7f50980757c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5098102a90 0x7f5098074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 --2- 172.21.15.37:0/2561007389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5098103450 0x7f5098075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 >> 172.21.15.37:0/2561007389 conn(0x7f5098079ba0 msgr2=0x7f50981018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 shutdown_connections 2024-04-25T21:13:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:00.812+0000 7f50a040f700 1 -- 172.21.15.37:0/2561007389 wait complete. 2024-04-25T21:13:01.693 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cd372993', seconds elapsed: 37.1 2024-04-25T21:13:01.693 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.8 2024-04-25T21:13:01.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:13:01.947 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:13:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:01 smithi037 ceph-mon[29195]: pgmap v139: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:13:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:01 smithi177 ceph-mon[33043]: pgmap v139: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:13:02.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:01 smithi098 ceph-mon[32569]: pgmap v139: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:13:03.787 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:03 smithi037 ceph-mon[29195]: pgmap v140: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T21:13:03.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 -- 172.21.15.37:0/1338420903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500102300 msgr2=0x7f850010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 --2- 172.21.15.37:0/1338420903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500102300 0x7f850010e7f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b0a0 tx=0x7f84f800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 -- 172.21.15.37:0/1338420903 shutdown_connections 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 --2- 172.21.15.37:0/1338420903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500102300 0x7f850010e7f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 --2- 172.21.15.37:0/1338420903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500107990 0x7f8500107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 --2- 172.21.15.37:0/1338420903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500101940 0x7f8500101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 -- 172.21.15.37:0/1338420903 >> 172.21.15.37:0/1338420903 conn(0x7f85000788f0 msgr2=0x7f8500078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 -- 172.21.15.37:0/1338420903 shutdown_connections 2024-04-25T21:13:03.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.792+0000 7f85067f0700 1 -- 172.21.15.37:0/1338420903 wait complete. 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 Processor -- start 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 -- start start 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 0x7f850019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500102300 0x7f850019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111b30 con 0x7f8500107990 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85001119b0 con 0x7f8500101940 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.793+0000 7f85067f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111cb0 con 0x7f8500102300 2024-04-25T21:13:03.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51328/0 (socket says 172.21.15.37:51328) 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 -- 172.21.15.37:0/2384244540 learned_addr learned my addr 172.21.15.37:0/2384244540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84fffff700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 0x7f850019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84ff7fe700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500102300 0x7f850019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 -- 172.21.15.37:0/2384244540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500102300 msgr2=0x7f850019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500102300 0x7f850019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 -- 172.21.15.37:0/2384244540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 msgr2=0x7f850019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 0x7f850019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 -- 172.21.15.37:0/2384244540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a02f0 con 0x7f8500107990 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84fffff700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 0x7f850019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f8504d8d700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b070 tx=0x7f84f8004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8021070 con 0x7f8500107990 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f8007e30 con 0x7f8500107990 2024-04-25T21:13:03.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80046d0 con 0x7f8500107990 2024-04-25T21:13:03.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a0580 con 0x7f8500107990 2024-04-25T21:13:03.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.794+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a7e60 con 0x7f8500107990 2024-04-25T21:13:03.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.795+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500103060 con 0x7f8500107990 2024-04-25T21:13:03.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.797+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84f8007980 con 0x7f8500107990 2024-04-25T21:13:03.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.797+0000 7f84fd7fa700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 0x7f84e80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:03.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.797+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f84f800f020 con 0x7f8500107990 2024-04-25T21:13:03.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.797+0000 7f84fffff700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 0x7f84e80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:03.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.798+0000 7f84fffff700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 0x7f84e80a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009fd0 tx=0x7f84f0009260 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:03.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:03.801+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84f802a020 con 0x7f8500107990 2024-04-25T21:13:04.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.054+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8500069730 con 0x7f8500107990 2024-04-25T21:13:04.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.055+0000 7f84fd7fa700 1 -- 172.21.15.37:0/2384244540 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8500069730 con 0x7f8500107990 2024-04-25T21:13:04.055 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:13:04.055 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:13:04.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 msgr2=0x7f84e80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:04.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 0x7f84e80a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009fd0 tx=0x7f84f0009260 comp rx=0 tx=0).stop 2024-04-25T21:13:04.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 msgr2=0x7f850019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b070 tx=0x7f84f8004cf0 comp rx=0 tx=0).stop 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 shutdown_connections 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f84e80a3e30 0x7f84e80a62f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8500107990 0x7f850019fbb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8500101940 0x7f850019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 --2- 172.21.15.37:0/2384244540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8500102300 0x7f850019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 >> 172.21.15.37:0/2384244540 conn(0x7f85000788f0 msgr2=0x7f850006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 shutdown_connections 2024-04-25T21:13:04.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:04.057+0000 7f85067f0700 1 -- 172.21.15.37:0/2384244540 wait complete. 2024-04-25T21:13:04.059 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:13:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:03 smithi177 ceph-mon[33043]: pgmap v140: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T21:13:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:03 smithi098 ceph-mon[32569]: pgmap v140: 97 pgs: 97 active+clean; 4.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-25T21:13:04.851 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 84.8 --await-for 42.0' 2024-04-25T21:13:04.851 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 84.8 --await-for 42.0 2024-04-25T21:13:04.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:04 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2384244540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:04 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2384244540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:05.106 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:13:05.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:04 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2384244540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:05 smithi037 ceph-mon[29195]: pgmap v141: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-25T21:13:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:05 smithi177 ceph-mon[33043]: pgmap v141: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-25T21:13:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:05 smithi098 ceph-mon[32569]: pgmap v141: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 30 op/s 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 -- 172.21.15.37:0/1235137402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 msgr2=0x7ff7a01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/1235137402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a01014d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff790009900 tx=0x7ff79002a410 comp rx=0 tx=0).stop 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 -- 172.21.15.37:0/1235137402 shutdown_connections 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/1235137402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 0x7ff7a010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/1235137402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a0101aa0 0x7ff7a0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/1235137402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a01014d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 -- 172.21.15.37:0/1235137402 >> 172.21.15.37:0/1235137402 conn(0x7ff7a0077fd0 msgr2=0x7ff7a00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 -- 172.21.15.37:0/1235137402 shutdown_connections 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.862+0000 7ff7a5b76640 1 -- 172.21.15.37:0/1235137402 wait complete. 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 Processor -- start 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 -- start start 2024-04-25T21:13:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 0x7ff7a010ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 0x7ff7a01a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01121f0 con 0x7ff7a01010f0 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0112070 con 0x7ff7a0106170 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff7a5b76640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0112370 con 0x7ff7a010f150 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51346/0 (socket says 172.21.15.37:51346) 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff79f7fe640 1 -- 172.21.15.37:0/2227569888 learned_addr learned my addr 172.21.15.37:0/2227569888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 0x7ff7a010ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.863+0000 7ff79ffff640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 0x7ff7a01a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 -- 172.21.15.37:0/2227569888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 msgr2=0x7ff7a01a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 0x7ff7a01a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 -- 172.21.15.37:0/2227569888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 msgr2=0x7ff7a010ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 0x7ff7a010ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 -- 172.21.15.37:0/2227569888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a01a6fc0 con 0x7ff7a01010f0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79ffff640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 0x7ff7a01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff790005940 tx=0x7ff7900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79d7fa640 1 -- 172.21.15.37:0/2227569888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79003f030 con 0x7ff7a01010f0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a01a7250 con 0x7ff7a01010f0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79d7fa640 1 -- 172.21.15.37:0/2227569888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff790003cb0 con 0x7ff7a01010f0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff79d7fa640 1 -- 172.21.15.37:0/2227569888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790035070 con 0x7ff7a01010f0 2024-04-25T21:13:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.864+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a01a7740 con 0x7ff7a01010f0 2024-04-25T21:13:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.866+0000 7ff79d7fa640 1 -- 172.21.15.37:0/2227569888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff790042030 con 0x7ff7a01010f0 2024-04-25T21:13:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.867+0000 7ff79d7fa640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 0x7ff78c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.867+0000 7ff79d7fa640 1 -- 172.21.15.37:0/2227569888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff7900f0cb0 con 0x7ff7a01010f0 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.867+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 0x7ff78c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 0x7ff78c0a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0110160 tx=0x7ff78800b390 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 Processor -- start 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 -- start start 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad32da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559cad292970 con 0x559cad32df40 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559cad22f4a0 con 0x559cad32d620 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff7a896f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559cad28e930 con 0x559cad338800 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51350/0 (socket says 172.21.15.37:51350) 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 -- 172.21.15.37:0/2556322244 learned_addr learned my addr 172.21.15.37:0/2556322244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff79ffff640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad32da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 -- 172.21.15.37:0/2556322244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 msgr2=0x559cad33abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 -- 172.21.15.37:0/2556322244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 msgr2=0x559cad32da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.868+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad32da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff796ffd640 1 -- 172.21.15.37:0/2556322244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cacfa5210 con 0x559cad32df40 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad32da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff788003c40 tx=0x7ff788013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4930aa880f5634fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff795ffb640 1 -- 172.21.15.37:0/2556322244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788011da0 con 0x559cad32df40 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff795ffb640 1 -- 172.21.15.37:0/2556322244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7880054d0 con 0x559cad32df40 2024-04-25T21:13:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.869+0000 7ff795ffb640 1 -- 172.21.15.37:0/2556322244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801e620 con 0x559cad32df40 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- 172.21.15.37:0/2556322244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 msgr2=0x559cad3382c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff788003c40 tx=0x7ff788013000 comp rx=0 tx=0).stop 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- 172.21.15.37:0/2556322244 shutdown_connections 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad3382c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2556322244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad32da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- 172.21.15.37:0/2556322244 >> 172.21.15.37:0/2556322244 conn(0x559cad329b30 msgr2=0x559cad329f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- 172.21.15.37:0/2556322244 shutdown_connections 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- 172.21.15.37:0/2556322244 wait complete. 2024-04-25T21:13:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.870+0000 7ff7a896f740 1 -- start start 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 Processor -- start 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad338160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad335450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559cad33b430 con 0x559cad338800 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559cad22d0b0 con 0x559cad32d620 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559cad33b5b0 con 0x559cad32df40 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff7a896f740 10 client.0 _ll_register_callbacks cb 0x559cad21ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51352/0 (socket says 172.21.15.37:51352) 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad338160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 -- 172.21.15.37:0/501385549 learned_addr learned my addr 172.21.15.37:0/501385549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff796ffd640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad335450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 -- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 msgr2=0x559cad335450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad335450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 -- 172.21.15.37:0/501385549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 msgr2=0x559cad338160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad338160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79ffff640 1 -- 172.21.15.37:0/501385549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cad071620 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.871+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad338160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff79ffff640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff798011a80 tx=0x7ff798011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ee8b2150282180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79801b400 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff79800e3b0 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7880054d0 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff79800e5b0 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 10 client.15945 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559cad347120 con 0x559cad338800 2024-04-25T21:13:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7880054d0 con 0x559cad338800 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff79801ba10 con 0x559cad338800 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 20 client.15945 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad348740 0x559cad34ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 4 client.15945 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.872+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cad2a0390 con 0x559cad348740 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.873+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad348740 0x559cad34ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.873+0000 7ff77effd640 10 client.15945 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.873+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad348740 0x559cad34ab30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=818663f236000a5b server_cookie=6dbf533ba0e73a84 in_seq=0 out_seq=0 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff77effd640 1 -- 172.21.15.37:0/501385549 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cad2a0390 con 0x559cad348740 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff77effd640 10 client.15945 handle_command_reply: tid=0 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 client.15945 shutdown 2024-04-25T21:13:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 msgr2=0x559cad3359b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff798011a80 tx=0x7ff798011f40 comp rx=0 tx=0).stop 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 shutdown_connections 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad348740 0x559cad34ab30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad338800 0x559cad3359b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad335450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 --2- 172.21.15.37:0/501385549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad338160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 >> 172.21.15.37:0/501385549 conn(0x559cad329b30 msgr2=0x559cad32f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff77effd640 0 client.15945 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 shutdown_connections 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 1 -- 172.21.15.37:0/501385549 wait complete. 2024-04-25T21:13:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.875+0000 7ff7a896f740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T21:13:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 Processor -- start 2024-04-25T21:13:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 -- start start 2024-04-25T21:13:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 0x559cad350440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad350980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad338800 0x559cad3407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01afd00 con 0x559cad32d620 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01b1840 con 0x559cad338800 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff7a896f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01b03d0 con 0x559cad32df40 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 0x559cad350440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 0x559cad350440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51354/0 (socket says 172.21.15.37:51354) 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad350980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.890+0000 7ff79f7fe640 1 -- 172.21.15.37:0/1016209520 learned_addr learned my addr 172.21.15.37:0/1016209520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff79ffff640 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad338800 0x559cad3407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 -- 172.21.15.37:0/1016209520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad338800 msgr2=0x559cad3407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad338800 0x559cad3407f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 -- 172.21.15.37:0/1016209520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 msgr2=0x559cad350440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 0x559cad350440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 -- 172.21.15.37:0/1016209520 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cad3339a0 con 0x559cad32df40 2024-04-25T21:13:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff796ffd640 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad350980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b470 tx=0x7ff788011ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c1b816cb361557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff7957fa640 1 -- 172.21.15.37:0/1016209520 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788005020 con 0x559cad32df40 2024-04-25T21:13:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff7957fa640 1 -- 172.21.15.37:0/1016209520 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff788004650 con 0x559cad32df40 2024-04-25T21:13:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.891+0000 7ff7957fa640 1 -- 172.21.15.37:0/1016209520 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78801e680 con 0x559cad32df40 2024-04-25T21:13:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- 172.21.15.37:0/1016209520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 msgr2=0x559cad350980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad350980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff78800b470 tx=0x7ff788011ba0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- 172.21.15.37:0/1016209520 shutdown_connections 2024-04-25T21:13:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad338800 0x559cad3407f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad32df40 0x559cad350980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 --2- 172.21.15.37:0/1016209520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32d620 0x559cad350440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- 172.21.15.37:0/1016209520 >> 172.21.15.37:0/1016209520 conn(0x559cad329b30 msgr2=0x559cad335410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- 172.21.15.37:0/1016209520 shutdown_connections 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- 172.21.15.37:0/1016209520 wait complete. 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.892+0000 7ff7a896f740 1 -- start start 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 Processor -- start 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad3405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01b1170 con 0x559cad32df40 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01b0aa0 con 0x559cad32d620 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01adce0 con 0x559cad338800 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff7a896f740 10 client.0 _ll_register_callbacks cb 0x7ff7980116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51358/0 (socket says 172.21.15.37:51358) 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 -- 172.21.15.37:0/2015435607 learned_addr learned my addr 172.21.15.37:0/2015435607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff79ffff640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad3405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 -- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 msgr2=0x559cad33d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 -- 172.21.15.37:0/2015435607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 msgr2=0x559cad3405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad3405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff796ffd640 1 -- 172.21.15.37:0/2015435607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cad33e1d0 con 0x559cad32df40 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff79ffff640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.893+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad3405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff796ffd640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff788033040 tx=0x7ff7880052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41137d45c8ea6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7880054a0 con 0x559cad32df40 2024-04-25T21:13:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff788004a10 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff788004650 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff788049050 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 10 client.15951 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559cad3ae7d0 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff788004650 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff78803ab60 con 0x559cad32df40 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 10 client.15951 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 20 client.15951 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad3aecf0 0x559cad3b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 4 client.15951 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.894+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cad29a200 con 0x559cad3aecf0 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.895+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad3aecf0 0x559cad3b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.895+0000 7ff79f7fe640 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad3aecf0 0x559cad3b10e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78645ec62984bc9e server_cookie=e75a31b2078432e6 in_seq=0 out_seq=0 2024-04-25T21:13:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.895+0000 7ff77effd640 10 client.15951 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff77effd640 1 -- 172.21.15.37:0/2015435607 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cad29a200 con 0x559cad3aecf0 2024-04-25T21:13:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff77effd640 10 client.15951 handle_command_reply: tid=0 2024-04-25T21:13:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 client.15951 shutdown 2024-04-25T21:13:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 msgr2=0x559cad340af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff788033040 tx=0x7ff7880052b0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 shutdown_connections 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559cad3aecf0 0x559cad3b10e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559cad338800 0x559cad33d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559cad32df40 0x559cad340af0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff77effd640 0 client.15951 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 --2- 172.21.15.37:0/2015435607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559cad32d620 0x559cad3405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 >> 172.21.15.37:0/2015435607 conn(0x559cad329b30 msgr2=0x559cad328420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 shutdown_connections 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 1 -- 172.21.15.37:0/2015435607 wait complete. 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.908+0000 7ff7a896f740 20 client.15951 trim_cache size 0 max 16384 2024-04-25T21:13:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:13:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:13:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:13:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:13:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 msgr2=0x7ff78c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 0x7ff78c0a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0110160 tx=0x7ff78800b390 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 msgr2=0x7ff7a010e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff790005940 tx=0x7ff7900042f0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 shutdown_connections 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff78c0a3df0 0x7ff78c0a62b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a010f150 0x7ff7a01a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7a0106170 0x7ff7a010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 --2- 172.21.15.37:0/2227569888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7a01010f0 0x7ff7a010e6d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 >> 172.21.15.37:0/2227569888 conn(0x7ff7a0077fd0 msgr2=0x7ff7a00ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 shutdown_connections 2024-04-25T21:13:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:06.909+0000 7ff7a5b76640 1 -- 172.21.15.37:0/2227569888 wait complete. 2024-04-25T21:13:06.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 -- 172.21.15.37:0/1978259775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c101970 msgr2=0x7f659c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/1978259775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c101970 0x7f659c101df0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6584009990 tx=0x7f6584009ca0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 -- 172.21.15.37:0/1978259775 shutdown_connections 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/1978259775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c102330 0x7f659c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/1978259775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f659c106840 0x7f659c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/1978259775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c101970 0x7f659c101df0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 -- 172.21.15.37:0/1978259775 >> 172.21.15.37:0/1978259775 conn(0x7f659c078980 msgr2=0x7f659c078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 -- 172.21.15.37:0/1978259775 shutdown_connections 2024-04-25T21:13:06.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.920+0000 7f65a2f2a700 1 -- 172.21.15.37:0/1978259775 wait complete. 2024-04-25T21:13:06.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 Processor -- start 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 -- start start 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c101970 0x7f659c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f659c106840 0x7f659c1a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111b10 con 0x7f659c102330 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111990 con 0x7f659c101970 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a2f2a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111c90 con 0x7f659c106840 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f659bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f659bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51380/0 (socket says 172.21.15.37:51380) 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f65a0cc6700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c101970 0x7f659c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.921+0000 7f659bfff700 1 -- 172.21.15.37:0/730722340 learned_addr learned my addr 172.21.15.37:0/730722340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 -- 172.21.15.37:0/730722340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f659c106840 msgr2=0x7f659c1a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f659c106840 0x7f659c1a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 -- 172.21.15.37:0/730722340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c101970 msgr2=0x7f659c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c101970 0x7f659c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 -- 172.21.15.37:0/730722340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c1a5350 con 0x7f659c102330 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f659bfff700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f65840057f0 tx=0x7f6584003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f6599ffb700 1 -- 172.21.15.37:0/730722340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584027070 con 0x7f659c102330 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f6599ffb700 1 -- 172.21.15.37:0/730722340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65840040a0 con 0x7f659c102330 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c1a5580 con 0x7f659c102330 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f6599ffb700 1 -- 172.21.15.37:0/730722340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584022e20 con 0x7f659c102330 2024-04-25T21:13:06.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.922+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a5a80 con 0x7f659c102330 2024-04-25T21:13:06.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.925+0000 7f6599ffb700 1 -- 172.21.15.37:0/730722340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f658402a070 con 0x7f659c102330 2024-04-25T21:13:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.926+0000 7f6599ffb700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 0x7f65880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.926+0000 7f6599ffb700 1 -- 172.21.15.37:0/730722340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f65840d1520 con 0x7f659c102330 2024-04-25T21:13:06.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.926+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 0x7f65880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.926+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 0x7f65880a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f658c00abd0 tx=0x7f658c00a380 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 Processor -- start 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 -- start start 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed6f5e20 con 0x55d6ed7adf00 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed69cac0 con 0x55d6ed7a3000 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed6f1de0 con 0x55d6ed7a26e0 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51396/0 (socket says 172.21.15.37:51396) 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65a14c7700 1 -- 172.21.15.37:0/1875314277 learned_addr learned my addr 172.21.15.37:0/1875314277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f659bfff700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.927+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 -- 172.21.15.37:0/1875314277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 msgr2=0x55d6ed7a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 -- 172.21.15.37:0/1875314277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 msgr2=0x55d6ed7b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 -- 172.21.15.37:0/1875314277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ed6608d0 con 0x55d6ed7a3000 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f65a14c7700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f659bfff700 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7ad9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65840cec40 tx=0x7f658400b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412fd0a4cf97c683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f6592ffd700 1 -- 172.21.15.37:0/1875314277 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658409d060 con 0x55d6ed7a3000 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f6592ffd700 1 -- 172.21.15.37:0/1875314277 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658409dc20 con 0x55d6ed7a3000 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.928+0000 7f6592ffd700 1 -- 172.21.15.37:0/1875314277 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658409ce80 con 0x55d6ed7a3000 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1875314277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 msgr2=0x55d6ed7ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7ad9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65840cec40 tx=0x7f658400b540 comp rx=0 tx=0).stop 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1875314277 shutdown_connections 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1875314277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7ad9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1875314277 >> 172.21.15.37:0/1875314277 conn(0x55d6ed6cfa20 msgr2=0x55d6ed79eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1875314277 shutdown_connections 2024-04-25T21:13:06.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1875314277 wait complete. 2024-04-25T21:13:06.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 -- start start 2024-04-25T21:13:06.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.929+0000 7f65ad7b3b80 1 Processor -- start 2024-04-25T21:13:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7adf00 0x55d6ed7aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed69cac0 con 0x55d6ed7a3000 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed6af740 con 0x55d6ed7a26e0 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d6ed7b0830 con 0x55d6ed7adf00 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65ad7b3b80 10 client.0 _ll_register_callbacks cb 0x55d6ed68eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51402/0 (socket says 172.21.15.37:51402) 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 -- 172.21.15.37:0/381977743 learned_addr learned my addr 172.21.15.37:0/381977743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f65a14c7700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7adf00 0x55d6ed7aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 -- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7adf00 msgr2=0x55d6ed7aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7adf00 0x55d6ed7aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 -- 172.21.15.37:0/381977743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 msgr2=0x55d6ed7ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 -- 172.21.15.37:0/381977743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ed643a60 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f659bfff700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f65840057c0 tx=0x7f658409d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d35be5ed1f4fa42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f657bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.930+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658409ce80 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f65840a5e80 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f658409dc20 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f658409d060 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 10 client.15963 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6ed7bafb0 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f658409dc20 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f65840d1040 con 0x55d6ed7a3000 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 10 client.15963 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 20 client.15963 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed7bd480 0x55d6ed7bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 4 client.15963 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.931+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6ed703610 con 0x55d6ed7bd480 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.932+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed7bd480 0x55d6ed7bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.932+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed7bd480 0x55d6ed7bf870 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0a39a3a275ace6 server_cookie=bdd78f2d2e967b93 in_seq=0 out_seq=0 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.932+0000 7f657bfff700 10 client.15963 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f657bfff700 1 -- 172.21.15.37:0/381977743 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6ed703610 con 0x55d6ed7bd480 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f657bfff700 10 client.15963 handle_command_reply: tid=0 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 client.15963 shutdown 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 msgr2=0x55d6ed7aa9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f65840057c0 tx=0x7f658409d9b0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 shutdown_connections 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed7bd480 0x55d6ed7bf870 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7adf00 0x55d6ed7aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/381977743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7a3000 0x55d6ed7aa9d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 >> 172.21.15.37:0/381977743 conn(0x55d6ed6cfa20 msgr2=0x55d6ed7a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f657bfff700 0 client.15963 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 shutdown_connections 2024-04-25T21:13:06.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/381977743 wait complete. 2024-04-25T21:13:06.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.933+0000 7f65ad7b3b80 20 client.15963 trim_cache size 0 max 16384 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 Processor -- start 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 -- start start 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 0x55d6ed7c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ad090 con 0x55d6ed7adf00 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1aebd0 con 0x55d6ed7a26e0 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ad760 con 0x55d6ed7a3000 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51414/0 (socket says 172.21.15.37:51414) 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 -- 172.21.15.37:0/3850051611 learned_addr learned my addr 172.21.15.37:0/3850051611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 -- 172.21.15.37:0/3850051611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 msgr2=0x55d6ed7c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f659bfff700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 0x55d6ed7c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 0x55d6ed7c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 -- 172.21.15.37:0/3850051611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 msgr2=0x55d6ed7c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a14c7700 1 -- 172.21.15.37:0/3850051611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6ed7bfd40 con 0x55d6ed7adf00 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.953+0000 7f659bfff700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 0x55d6ed7c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65a14c7700 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6594011210 tx=0x7f65940089c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e58ba462d24eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65927fc700 1 -- 172.21.15.37:0/3850051611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659401b020 con 0x55d6ed7adf00 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65927fc700 1 -- 172.21.15.37:0/3850051611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6594011d60 con 0x55d6ed7adf00 2024-04-25T21:13:06.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65927fc700 1 -- 172.21.15.37:0/3850051611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594017770 con 0x55d6ed7adf00 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/3850051611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 msgr2=0x55d6ed7b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6594011210 tx=0x7f65940089c0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/3850051611 shutdown_connections 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b5ab0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/3850051611 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a3000 0x55d6ed7c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/3850051611 >> 172.21.15.37:0/3850051611 conn(0x55d6ed6cfa20 msgr2=0x55d6ed7b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.954+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/3850051611 shutdown_connections 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/3850051611 wait complete. 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 -- start start 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 Processor -- start 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ae500 con 0x55d6ed7adf00 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ade30 con 0x55d6ed7a3000 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1ab050 con 0x55d6ed7a26e0 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.955+0000 7f65ad7b3b80 10 client.0 _ll_register_callbacks cb 0x55d6ed68eab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51430/0 (socket says 172.21.15.37:51430) 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 -- 172.21.15.37:0/1520256240 learned_addr learned my addr 172.21.15.37:0/1520256240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f659bfff700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 -- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 msgr2=0x55d6ed7b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 -- 172.21.15.37:0/1520256240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 msgr2=0x55d6ed7b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 -- 172.21.15.37:0/1520256240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6ed7b2a70 con 0x55d6ed7adf00 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f659bfff700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65a14c7700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6594004b10 tx=0x7f6594016710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29560408f73a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f657bfff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594008560 con 0x55d6ed7adf00 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6594011b60 con 0x55d6ed7adf00 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6594011d60 con 0x55d6ed7adf00 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65940169e0 con 0x55d6ed7adf00 2024-04-25T21:13:06.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65ad7b3b80 10 client.15969 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.956+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d6ed8289a0 con 0x55d6ed7adf00 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6594011d60 con 0x55d6ed7adf00 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6594033450 con 0x55d6ed7adf00 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 10 client.15969 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 20 client.15969 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed829030 0x55d6ed82b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 4 client.15969 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.8, "await_for": 42.0}] 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.8, "await_for": 42.0}) -- 0x55d6ed6fd6b0 con 0x55d6ed829030 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.957+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed829030 0x55d6ed82b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.958+0000 7f657bfff700 10 client.15969 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.958+0000 7f65a0cc6700 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed829030 0x55d6ed82b450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fefc568ad53dfcde server_cookie=fbc238de29422032 in_seq=0 out_seq=0 2024-04-25T21:13:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f657bfff700 1 -- 172.21.15.37:0/1520256240 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d6ed6fd6b0 con 0x55d6ed829030 2024-04-25T21:13:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f657bfff700 10 client.15969 handle_command_reply: tid=0 2024-04-25T21:13:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 client.15969 shutdown 2024-04-25T21:13:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 msgr2=0x55d6ed7b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6594004b10 tx=0x7f6594016710 comp rx=0 tx=0).stop 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 shutdown_connections 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d6ed829030 0x55d6ed82b450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d6ed7adf00 0x55d6ed7b24b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f657bfff700 0 client.15969 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d6ed7a26e0 0x55d6ed7b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 --2- 172.21.15.37:0/1520256240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d6ed7a3000 0x55d6ed7b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 >> 172.21.15.37:0/1520256240 conn(0x55d6ed6cfa20 msgr2=0x55d6ed79e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 shutdown_connections 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 1 -- 172.21.15.37:0/1520256240 wait complete. 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.974+0000 7f65ad7b3b80 20 client.15969 trim_cache size 0 max 16384 2024-04-25T21:13:06.975 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 14, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "d1f85f6c": { 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 14, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.8, 2024-04-25T21:13:06.976 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:06.977 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 msgr2=0x7f65880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 0x7f65880a6390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f658c00abd0 tx=0x7f658c00a380 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 msgr2=0x7f659c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f65840057f0 tx=0x7f6584003680 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 shutdown_connections 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f65880a3ed0 0x7f65880a6390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f659c106840 0x7f659c1a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f659c101970 0x7f659c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 --2- 172.21.15.37:0/730722340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f659c102330 0x7f659c19a2d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 >> 172.21.15.37:0/730722340 conn(0x7f659c078980 msgr2=0x7f659c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 shutdown_connections 2024-04-25T21:13:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:06.975+0000 7f65a2f2a700 1 -- 172.21.15.37:0/730722340 wait complete. 2024-04-25T21:13:07.794 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd1f85f6c', quiesce took 6.1 seconds. Will release after: 24.8 2024-04-25T21:13:07.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:07 smithi037 ceph-mon[29195]: pgmap v142: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:07 smithi177 ceph-mon[33043]: pgmap v142: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:07 smithi098 ceph-mon[32569]: pgmap v142: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:09 smithi037 ceph-mon[29195]: pgmap v143: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:09 smithi177 ceph-mon[33043]: pgmap v143: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:09 smithi098 ceph-mon[32569]: pgmap v143: 97 pgs: 97 active+clean; 4.2 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 KiB/s wr, 24 op/s 2024-04-25T21:13:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:11 smithi037 ceph-mon[29195]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:11 smithi177 ceph-mon[33043]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:11 smithi098 ceph-mon[32569]: pgmap v144: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:13.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:13 smithi037 ceph-mon[29195]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-25T21:13:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:13 smithi177 ceph-mon[33043]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-25T21:13:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:13 smithi098 ceph-mon[32569]: pgmap v145: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-25T21:13:15.968 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:15 smithi098 ceph-mon[32569]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:15 smithi037 ceph-mon[29195]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:15 smithi177 ceph-mon[33043]: pgmap v146: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 47 KiB/s wr, 33 op/s 2024-04-25T21:13:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:17 smithi037 ceph-mon[29195]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:17 smithi177 ceph-mon[33043]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:17 smithi098 ceph-mon[32569]: pgmap v147: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:19 smithi037 ceph-mon[29195]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 -- 172.21.15.37:0/2794367907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38074ca0 msgr2=0x7f6a3810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2794367907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38074ca0 0x7f6a3810e230 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009900 tx=0x7f6a2c02a480 comp rx=0 tx=0).stop 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 -- 172.21.15.37:0/2794367907 shutdown_connections 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2794367907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a3810e770 0x7f6a38110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2794367907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a38074ca0 0x7f6a3810e230 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2794367907 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 0x7f6a38074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 -- 172.21.15.37:0/2794367907 >> 172.21.15.37:0/2794367907 conn(0x7f6a380fda00 msgr2=0x7f6a380ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 -- 172.21.15.37:0/2794367907 shutdown_connections 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.008+0000 7f6a406b9640 1 -- 172.21.15.37:0/2794367907 wait complete. 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 Processor -- start 2024-04-25T21:13:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 -- start start 2024-04-25T21:13:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 0x7f6a381a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 0x7f6a381a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381143b0 con 0x7f6a3810e770 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38114230 con 0x7f6a38074ca0 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a406b9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38114530 con 0x7f6a38074380 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a3ec56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a3ec56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51864/0 (socket says 172.21.15.37:51864) 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a3ec56640 1 -- 172.21.15.37:0/2276506469 learned_addr learned my addr 172.21.15.37:0/2276506469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.009+0000 7f6a3e455640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 0x7f6a381a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 0x7f6a381a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 -- 172.21.15.37:0/2276506469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 msgr2=0x7f6a381a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 0x7f6a381a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 -- 172.21.15.37:0/2276506469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 msgr2=0x7f6a381a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 0x7f6a381a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 -- 172.21.15.37:0/2276506469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3819b010 con 0x7f6a3810e770 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 0x7f6a381a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a3ec56640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800eaa0 tx=0x7f6a2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a377fe640 1 -- 172.21.15.37:0/2276506469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800cdb0 con 0x7f6a3810e770 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3819b300 con 0x7f6a3810e770 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a377fe640 1 -- 172.21.15.37:0/2276506469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a28005020 con 0x7f6a3810e770 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a377fe640 1 -- 172.21.15.37:0/2276506469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280116d0 con 0x7f6a3810e770 2024-04-25T21:13:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.010+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a7610 con 0x7f6a3810e770 2024-04-25T21:13:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.012+0000 7f6a377fe640 1 -- 172.21.15.37:0/2276506469 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a280051d0 con 0x7f6a3810e770 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 Processor -- start 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 -- start start 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f264a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f271bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55834f1c9970 con 0x55834f264f40 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55834f1664a0 con 0x55834f264620 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a433cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55834f1c5930 con 0x55834f26f800 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a377fe640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 0x7f6a200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51876/0 (socket says 172.21.15.37:51876) 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a3e455640 1 -- 172.21.15.37:0/1702139855 learned_addr learned my addr 172.21.15.37:0/1702139855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.013+0000 7f6a3ec56640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 0x7f6a200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f271bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f264a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a377fe640 1 -- 172.21.15.37:0/2276506469 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a280d0930 con 0x7f6a3810e770 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 -- 172.21.15.37:0/1702139855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 msgr2=0x55834f271bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f271bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 -- 172.21.15.37:0/1702139855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 msgr2=0x55834f264a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f264a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 -- 172.21.15.37:0/1702139855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834eedc210 con 0x55834f264f40 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f271bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b760 tx=0x7f6a3000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97a41d3cd095f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a3ec56640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 0x7f6a200a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800cfd0 tx=0x7f6a280b1040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a34ff9640 1 -- 172.21.15.37:0/1702139855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30003dd0 con 0x55834f264f40 2024-04-25T21:13:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a34ff9640 1 -- 172.21.15.37:0/1702139855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a30015070 con 0x55834f264f40 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.014+0000 7f6a34ff9640 1 -- 172.21.15.37:0/1702139855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004d90 con 0x55834f264f40 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- 172.21.15.37:0/1702139855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 msgr2=0x55834f26f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a3000b760 tx=0x7f6a3000bc20 comp rx=0 tx=0).stop 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- 172.21.15.37:0/1702139855 shutdown_connections 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f271bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26f2c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1702139855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f264a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- 172.21.15.37:0/1702139855 >> 172.21.15.37:0/1702139855 conn(0x55834f260b30 msgr2=0x55834f260f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- 172.21.15.37:0/1702139855 shutdown_connections 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- 172.21.15.37:0/1702139855 wait complete. 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 -- start start 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.015+0000 7f6a433cd740 1 Processor -- start 2024-04-25T21:13:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264f40 0x55834f26c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f26c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55834f272430 con 0x55834f264620 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55834f1640b0 con 0x55834f264f40 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55834f2725b0 con 0x55834f26f800 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a433cd740 10 client.0 _ll_register_callbacks cb 0x55834f151d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51884/0 (socket says 172.21.15.37:51884) 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 -- 172.21.15.37:0/52935917 learned_addr learned my addr 172.21.15.37:0/52935917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3e455640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264f40 0x55834f26c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 -- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 msgr2=0x55834f26c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f26c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 -- 172.21.15.37:0/52935917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264f40 msgr2=0x55834f26c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264f40 0x55834f26c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 -- 172.21.15.37:0/52935917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834efa8620 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009900 tx=0x7f6a2c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a6f8d8319d309f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.016+0000 7f6a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c044070 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a2c004240 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a30015070 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c050050 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 10 client.15981 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55834f27e310 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a30015070 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a2c03a040 con 0x55834f264620 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 10 client.15981 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 20 client.15981 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f27f780 0x55834f281bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 4 client.15981 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.017+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55834f1d7390 con 0x55834f27f780 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.018+0000 7f6a3e455640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f27f780 0x55834f281bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.018+0000 7f6a3e455640 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f27f780 0x55834f281bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39683be3eec7ba17 server_cookie=cf12578f3e1ee13a in_seq=0 out_seq=0 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.018+0000 7f6a1dffb640 10 client.15981 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a1dffb640 1 -- 172.21.15.37:0/52935917 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55834f1d7390 con 0x55834f27f780 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a1dffb640 10 client.15981 handle_command_reply: tid=0 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 client.15981 shutdown 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 msgr2=0x55834f26f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009900 tx=0x7f6a2c005dc0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 shutdown_connections 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f27f780 0x55834f281bb0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f26c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264f40 0x55834f26c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a1dffb640 0 client.15981 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 --2- 172.21.15.37:0/52935917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264620 0x55834f26f120 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 >> 172.21.15.37:0/52935917 conn(0x55834f260b30 msgr2=0x55834f2668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 shutdown_connections 2024-04-25T21:13:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 1 -- 172.21.15.37:0/52935917 wait complete. 2024-04-25T21:13:20.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.019+0000 7f6a433cd740 20 client.15981 trim_cache size 0 max 16384 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 Processor -- start 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 -- start start 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f27b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f2874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381afd50 con 0x55834f264f40 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b1890 con 0x55834f264620 2024-04-25T21:13:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a433cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b0420 con 0x55834f26f800 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51898/0 (socket says 172.21.15.37:51898) 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 -- 172.21.15.37:0/2906959650 learned_addr learned my addr 172.21.15.37:0/2906959650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f27b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 -- 172.21.15.37:0/2906959650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 msgr2=0x55834f2874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f2874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 -- 172.21.15.37:0/2906959650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 msgr2=0x55834f27b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f27b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 -- 172.21.15.37:0/2906959650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834f261700 con 0x55834f264f40 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.035+0000 7f6a3e455640 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004940 tx=0x7f6a30004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5fad0129312c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a1ffff640 1 -- 172.21.15.37:0/2906959650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30003e50 con 0x55834f264f40 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a1ffff640 1 -- 172.21.15.37:0/2906959650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a30015050 con 0x55834f264f40 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a1ffff640 1 -- 172.21.15.37:0/2906959650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30011860 con 0x55834f264f40 2024-04-25T21:13:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 -- 172.21.15.37:0/2906959650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 msgr2=0x55834f26ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004940 tx=0x7f6a30004970 comp rx=0 tx=0).stop 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 -- 172.21.15.37:0/2906959650 shutdown_connections 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f26f800 0x55834f2874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f26ec50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 --2- 172.21.15.37:0/2906959650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f264620 0x55834f27b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 -- 172.21.15.37:0/2906959650 >> 172.21.15.37:0/2906959650 conn(0x55834f260b30 msgr2=0x55834f2679d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 -- 172.21.15.37:0/2906959650 shutdown_connections 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.036+0000 7f6a433cd740 1 -- 172.21.15.37:0/2906959650 wait complete. 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 -- start start 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 Processor -- start 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f264620 0x55834f277750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 0x55834f272ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b11c0 con 0x55834f264f40 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b0af0 con 0x55834f26f800 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381add30 con 0x55834f264620 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a433cd740 10 client.0 _ll_register_callbacks cb 0x7f6a2c032ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51912/0 (socket says 172.21.15.37:51912) 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 -- 172.21.15.37:0/1410840431 learned_addr learned my addr 172.21.15.37:0/1410840431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 0x55834f272ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 -- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f264620 msgr2=0x55834f277750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f264620 0x55834f277750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 -- 172.21.15.37:0/1410840431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 msgr2=0x55834f272ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 0x55834f272ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3e455640 1 -- 172.21.15.37:0/1410840431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834f2734a0 con 0x55834f264f40 2024-04-25T21:13:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.037+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 0x55834f272ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a3e455640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004940 tx=0x7f6a30003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286305f358abbdf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30034060 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6a30027320 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a30015050 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6a30003e60 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 10 client.15987 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55834f2e58c0 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a30015050 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6a30011cd0 con 0x55834f264f40 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 20 client.15987 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f2e5d90 0x55834f2e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 4 client.15987 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:13:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.038+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55834f1d1200 con 0x55834f2e5d90 2024-04-25T21:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.039+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f2e5d90 0x55834f2e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.039+0000 7f6a3dc54640 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f2e5d90 0x55834f2e81d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4afa11dfa39a2ca server_cookie=d0a691f68cdabb44 in_seq=0 out_seq=0 2024-04-25T21:13:20.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.039+0000 7f6a1dffb640 10 client.15987 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a1dffb640 1 -- 172.21.15.37:0/1410840431 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55834f1d1200 con 0x55834f2e5d90 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a1dffb640 10 client.15987 handle_command_reply: tid=0 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 client.15987 shutdown 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 msgr2=0x55834f277c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004940 tx=0x7f6a30003a70 comp rx=0 tx=0).stop 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 shutdown_connections 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55834f2e5d90 0x55834f2e81d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55834f26f800 0x55834f272ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55834f264f40 0x55834f277c90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a1dffb640 0 client.15987 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 --2- 172.21.15.37:0/1410840431 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55834f264620 0x55834f277750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 >> 172.21.15.37:0/1410840431 conn(0x55834f260b30 msgr2=0x55834f25f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 shutdown_connections 2024-04-25T21:13:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 1 -- 172.21.15.37:0/1410840431 wait complete. 2024-04-25T21:13:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.052+0000 7f6a433cd740 20 client.15987 trim_cache size 0 max 16384 2024-04-25T21:13:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:13:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:13:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:13:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 msgr2=0x7f6a200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 0x7f6a200a6280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800cfd0 tx=0x7f6a280b1040 comp rx=0 tx=0).stop 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 msgr2=0x7f6a3819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800eaa0 tx=0x7f6a2800ef60 comp rx=0 tx=0).stop 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 shutdown_connections 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3810e770 0x7f6a3819a750 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a38074ca0 0x7f6a381a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6a200a3dc0 0x7f6a200a6280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 --2- 172.21.15.37:0/2276506469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a38074380 0x7f6a381a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 >> 172.21.15.37:0/2276506469 conn(0x7f6a380fda00 msgr2=0x7f6a380fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 shutdown_connections 2024-04-25T21:13:20.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:20.053+0000 7f6a406b9640 1 -- 172.21.15.37:0/2276506469 wait complete. 2024-04-25T21:13:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:19 smithi177 ceph-mon[33043]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:20.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:19 smithi098 ceph-mon[32569]: pgmap v148: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:13:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:21 smithi037 ceph-mon[29195]: pgmap v149: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T21:13:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:21 smithi177 ceph-mon[33043]: pgmap v149: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T21:13:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:21 smithi098 ceph-mon[32569]: pgmap v149: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-25T21:13:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:23 smithi037 ceph-mon[29195]: pgmap v150: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T21:13:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:23 smithi177 ceph-mon[33043]: pgmap v150: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T21:13:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:23 smithi098 ceph-mon[32569]: pgmap v150: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-04-25T21:13:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:25 smithi037 ceph-mon[29195]: pgmap v151: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:13:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:25 smithi177 ceph-mon[33043]: pgmap v151: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:13:26.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:25 smithi098 ceph-mon[32569]: pgmap v151: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:13:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:27 smithi037 ceph-mon[29195]: pgmap v152: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:27 smithi177 ceph-mon[33043]: pgmap v152: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:27 smithi098 ceph-mon[32569]: pgmap v152: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:29 smithi037 ceph-mon[29195]: pgmap v153: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:30.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:29 smithi177 ceph-mon[33043]: pgmap v153: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:29 smithi098 ceph-mon[32569]: pgmap v153: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-25T21:13:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:31 smithi037 ceph-mon[29195]: pgmap v154: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 15 op/s 2024-04-25T21:13:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:31 smithi177 ceph-mon[33043]: pgmap v154: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 15 op/s 2024-04-25T21:13:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:31 smithi098 ceph-mon[32569]: pgmap v154: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 15 op/s 2024-04-25T21:13:32.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:13:32.855 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1688406984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2641011a0 msgr2=0x7ff264101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1688406984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2641011a0 0x7ff264101620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff25402a400 comp rx=0 tx=0).stop 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1688406984 shutdown_connections 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1688406984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff264101b60 0x7ff26410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1688406984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2641011a0 0x7ff264101620 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1688406984 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264106fb0 0x7ff264107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1688406984 >> 172.21.15.37:0/1688406984 conn(0x7ff264076c50 msgr2=0x7ff264077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.144+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1688406984 shutdown_connections 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1688406984 wait complete. 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 Processor -- start 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 -- start start 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2641011a0 0x7ff2641974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264101b60 0x7ff2641979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff264111060 con 0x7ff264106fb0 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff264110ee0 con 0x7ff2641011a0 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26c2cf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641111e0 con 0x7ff264101b60 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26a86c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26a86c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59386/0 (socket says 172.21.15.37:59386) 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26a86c640 1 -- 172.21.15.37:0/1353888952 learned_addr learned my addr 172.21.15.37:0/1353888952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.145+0000 7ff26a06b640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2641011a0 0x7ff2641974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26986a640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264101b60 0x7ff2641979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 -- 172.21.15.37:0/1353888952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264101b60 msgr2=0x7ff2641979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264101b60 0x7ff2641979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 -- 172.21.15.37:0/1353888952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2641011a0 msgr2=0x7ff2641974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2641011a0 0x7ff2641974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 -- 172.21.15.37:0/1353888952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26419c490 con 0x7ff264106fb0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26a86c640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eaa0 tx=0x7ff25c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff25b7fe640 1 -- 172.21.15.37:0/1353888952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004d20 con 0x7ff264106fb0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff25b7fe640 1 -- 172.21.15.37:0/1353888952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25c005020 con 0x7ff264106fb0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26419c780 con 0x7ff264106fb0 2024-04-25T21:13:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff25b7fe640 1 -- 172.21.15.37:0/1353888952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0116b0 con 0x7ff264106fb0 2024-04-25T21:13:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.146+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26406a780 con 0x7ff264106fb0 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.148+0000 7ff25b7fe640 1 -- 172.21.15.37:0/1353888952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff25c0051d0 con 0x7ff264106fb0 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 Processor -- start 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 -- start start 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dc4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 0x55caf7dd1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55caf7d29970 con 0x55caf7dc4f40 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55caf7cc64a0 con 0x55caf7dc4620 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26efed740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55caf7d25930 con 0x55caf7dcf800 2024-04-25T21:13:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26a06b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff25b7fe640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 0x7ff2500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26a06b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59388/0 (socket says 172.21.15.37:59388) 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26a06b640 1 -- 172.21.15.37:0/620689415 learned_addr learned my addr 172.21.15.37:0/620689415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26986a640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dc4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.149+0000 7ff26a86c640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 0x7ff2500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26986a640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 0x55caf7dd1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 -- 172.21.15.37:0/620689415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 msgr2=0x55caf7dd1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 0x55caf7dd1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 -- 172.21.15.37:0/620689415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 msgr2=0x55caf7dc4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dc4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 -- 172.21.15.37:0/620689415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf7a3c210 con 0x55caf7dc4f40 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff25b7fe640 1 -- 172.21.15.37:0/1353888952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff25c0d0a10 con 0x7ff264106fb0 2024-04-25T21:13:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26986a640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 0x55caf7dd1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a06b640 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad00 tx=0x7ff26000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1349aa65c57dc8ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff26a86c640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 0x7ff2500a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009270 tx=0x7ff25c06e3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff258bfe640 1 -- 172.21.15.37:0/620689415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26001d070 con 0x55caf7dc4f40 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff258bfe640 1 -- 172.21.15.37:0/620689415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff260018af0 con 0x55caf7dc4f40 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.150+0000 7ff258bfe640 1 -- 172.21.15.37:0/620689415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260010b30 con 0x55caf7dc4f40 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- 172.21.15.37:0/620689415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 msgr2=0x55caf7dcf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff26000ad00 tx=0x7ff26000f790 comp rx=0 tx=0).stop 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- 172.21.15.37:0/620689415 shutdown_connections 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dcf800 0x55caf7dd1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dcf2c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- 172.21.15.37:0/620689415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dc4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- 172.21.15.37:0/620689415 >> 172.21.15.37:0/620689415 conn(0x55caf7dc0b30 msgr2=0x55caf7dc0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- 172.21.15.37:0/620689415 shutdown_connections 2024-04-25T21:13:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- 172.21.15.37:0/620689415 wait complete. 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 -- start start 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 Processor -- start 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dcf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4f40 0x55caf7dcc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.151+0000 7ff26efed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 0x55caf7dcc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26efed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55caf7dd2430 con 0x55caf7dcf800 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26efed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55caf7cc40b0 con 0x55caf7dc4620 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26efed740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55caf7dd25b0 con 0x55caf7dc4f40 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26efed740 10 client.0 _ll_register_callbacks cb 0x55caf7cb1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dcf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 0x55caf7dcc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dcf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:58132/0 (socket says 172.21.15.37:58132) 2024-04-25T21:13:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 -- 172.21.15.37:0/1296379595 learned_addr learned my addr 172.21.15.37:0/1296379595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26a06b640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4f40 0x55caf7dcc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 -- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4f40 msgr2=0x55caf7dcc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4f40 0x55caf7dcc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 -- 172.21.15.37:0/1296379595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 msgr2=0x55caf7dcf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dcf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 -- 172.21.15.37:0/1296379595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf7b08620 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26986a640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 0x55caf7dcc900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff254005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665dc76f142b2455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254035070 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff260018af0 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.152+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff254004030 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff254052050 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 10 client.15999 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caf7dde240 con 0x55caf7dcf800 2024-04-25T21:13:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff260018af0 con 0x55caf7dcf800 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2540035f0 con 0x55caf7dcf800 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 20 client.15999 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7ddf6b0 0x55caf7de1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 4 client.15999 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf7d37390 con 0x55caf7ddf6b0 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.153+0000 7ff26a06b640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7ddf6b0 0x55caf7de1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.154+0000 7ff24dffb640 10 client.15999 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.154+0000 7ff26a06b640 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7ddf6b0 0x55caf7de1aa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3dd2df70c7763ac server_cookie=fb78d92f8817744 in_seq=0 out_seq=0 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.154+0000 7ff24dffb640 1 -- 172.21.15.37:0/1296379595 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caf7d37390 con 0x55caf7ddf6b0 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.154+0000 7ff24dffb640 10 client.15999 handle_command_reply: tid=0 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 client.15999 shutdown 2024-04-25T21:13:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 msgr2=0x55caf7dcc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 0x55caf7dcc900 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff254009900 tx=0x7ff254005db0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 shutdown_connections 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7ddf6b0 0x55caf7de1aa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dcf800 0x55caf7dcc900 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4f40 0x55caf7dcc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 --2- 172.21.15.37:0/1296379595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dcf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff24dffb640 0 client.15999 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 >> 172.21.15.37:0/1296379595 conn(0x55caf7dc0b30 msgr2=0x55caf7dc68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 shutdown_connections 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 1 -- 172.21.15.37:0/1296379595 wait complete. 2024-04-25T21:13:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.155+0000 7ff26efed740 20 client.15999 trim_cache size 0 max 16384 2024-04-25T21:13:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 Processor -- start 2024-04-25T21:13:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 -- start start 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 0x55caf7dc7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4f40 0x55caf7dcec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ada10 con 0x55caf7dc0b30 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641af550 con 0x55caf7dc4f40 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26efed740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ae0e0 con 0x55caf7dc4620 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59404/0 (socket says 172.21.15.37:59404) 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 -- 172.21.15.37:0/2792213485 learned_addr learned my addr 172.21.15.37:0/2792213485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26a06b640 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 0x55caf7dc7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 -- 172.21.15.37:0/2792213485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 msgr2=0x55caf7dc7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 0x55caf7dc7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 -- 172.21.15.37:0/2792213485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4f40 msgr2=0x55caf7dcec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4f40 0x55caf7dcec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 -- 172.21.15.37:0/2792213485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf7dca950 con 0x55caf7dc0b30 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26a06b640 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 0x55caf7dc7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.170+0000 7ff26986a640 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff254050040 tx=0x7ff254005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d427e3bfbf4714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff24ffff640 1 -- 172.21.15.37:0/2792213485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254035040 con 0x55caf7dc0b30 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff24ffff640 1 -- 172.21.15.37:0/2792213485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25403ce40 con 0x55caf7dc0b30 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff24ffff640 1 -- 172.21.15.37:0/2792213485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff254003df0 con 0x55caf7dc0b30 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 -- 172.21.15.37:0/2792213485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 msgr2=0x55caf7ddb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff254050040 tx=0x7ff254005760 comp rx=0 tx=0).stop 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 -- 172.21.15.37:0/2792213485 shutdown_connections 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4f40 0x55caf7dcec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc4620 0x55caf7dc7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 --2- 172.21.15.37:0/2792213485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc0b30 0x55caf7ddb2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 -- 172.21.15.37:0/2792213485 >> 172.21.15.37:0/2792213485 conn(0x55caf7dcca60 msgr2=0x55caf7dc7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 -- 172.21.15.37:0/2792213485 shutdown_connections 2024-04-25T21:13:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.171+0000 7ff26efed740 1 -- 172.21.15.37:0/2792213485 wait complete. 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 -- start start 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 Processor -- start 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc0b30 0x55caf7dd7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dd2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641aee80 con 0x55caf7dc4f40 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ae7b0 con 0x55caf7dc4620 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7ff2641ab9f0 con 0x55caf7dc0b30 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26efed740 10 client.0 _ll_register_callbacks cb 0x7ff254031670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59416/0 (socket says 172.21.15.37:59416) 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 -- 172.21.15.37:0/3514704333 learned_addr learned my addr 172.21.15.37:0/3514704333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc0b30 0x55caf7dd7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26a06b640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dd2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 -- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc0b30 msgr2=0x55caf7dd7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc0b30 0x55caf7dd7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 -- 172.21.15.37:0/3514704333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 msgr2=0x55caf7dd2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dd2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 -- 172.21.15.37:0/3514704333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf7dd3bc0 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26a06b640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dd2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff26986a640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff254003a90 tx=0x7ff254003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f07a76e42e60e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.172+0000 7ff24dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25403cab0 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7ff254003b90 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25403ce40 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff25403cc50 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55caf7e45ad0 con 0x55caf7dc4f40 2024-04-25T21:13:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff25403ce40 con 0x55caf7dc4f40 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff254062020 con 0x55caf7dc4f40 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 20 client.16005 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7dcf800 0x55caf7e48170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 4 client.16005 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55caf7d31200 con 0x55caf7dcf800 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.173+0000 7ff26a06b640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7dcf800 0x55caf7e48170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.174+0000 7ff24dffb640 10 client.16005 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.174+0000 7ff26a06b640 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7dcf800 0x55caf7e48170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9ea3bb69a60dbf server_cookie=4d060ec9dd52f9ff in_seq=0 out_seq=0 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff24dffb640 1 -- 172.21.15.37:0/3514704333 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55caf7d31200 con 0x55caf7dcf800 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff24dffb640 10 client.16005 handle_command_reply: tid=0 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 client.16005 shutdown 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 msgr2=0x55caf7dd33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff254003a90 tx=0x7ff254003800 comp rx=0 tx=0).stop 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 shutdown_connections 2024-04-25T21:13:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55caf7dcf800 0x55caf7e48170 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caf7dc4f40 0x55caf7dd33c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff24dffb640 0 client.16005 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55caf7dc4620 0x55caf7dd2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 --2- 172.21.15.37:0/3514704333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55caf7dc0b30 0x55caf7dd7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 >> 172.21.15.37:0/3514704333 conn(0x55caf7dcca60 msgr2=0x55caf7dbfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.184+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 shutdown_connections 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26efed740 1 -- 172.21.15.37:0/3514704333 wait complete. 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26efed740 20 client.16005 trim_cache size 0 max 16384 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:13:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:13:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:13:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:13:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 msgr2=0x7ff2500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 0x7ff2500a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009270 tx=0x7ff25c06e3c0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 msgr2=0x7ff26419bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00eaa0 tx=0x7ff25c00eea0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 shutdown_connections 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff264106fb0 0x7ff26419bd60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7ff2500a3dc0 0x7ff2500a6280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff264101b60 0x7ff2641979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 --2- 172.21.15.37:0/1353888952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2641011a0 0x7ff2641974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 >> 172.21.15.37:0/1353888952 conn(0x7ff264076c50 msgr2=0x7ff2640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 shutdown_connections 2024-04-25T21:13:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:33.185+0000 7ff26c2cf640 1 -- 172.21.15.37:0/1353888952 wait complete. 2024-04-25T21:13:33.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:33 smithi037 ceph-mon[29195]: pgmap v155: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-25T21:13:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:33 smithi177 ceph-mon[33043]: pgmap v155: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-25T21:13:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:33 smithi098 ceph-mon[32569]: pgmap v155: 97 pgs: 97 active+clean; 4.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-25T21:13:34.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 -- 172.21.15.37:0/3015000797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244108a70 msgr2=0x7f8244108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 --2- 172.21.15.37:0/3015000797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244108a70 0x7f8244108e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8238009990 tx=0x7f8238009ca0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 -- 172.21.15.37:0/3015000797 shutdown_connections 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 --2- 172.21.15.37:0/3015000797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244103480 0x7f824410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 --2- 172.21.15.37:0/3015000797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244108a70 0x7f8244108e50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 --2- 172.21.15.37:0/3015000797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 0x7f8244102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 -- 172.21.15.37:0/3015000797 >> 172.21.15.37:0/3015000797 conn(0x7f8244079bb0 msgr2=0x7f8244079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 -- 172.21.15.37:0/3015000797 shutdown_connections 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.939+0000 7f824c00e700 1 -- 172.21.15.37:0/3015000797 wait complete. 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 Processor -- start 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 -- start start 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 0x7f8244074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 0x7f82440757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112c00 con 0x7f8244103480 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112a80 con 0x7f8244108a70 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f824c00e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112d80 con 0x7f8244102ac0 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f82495a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f82495a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59440/0 (socket says 172.21.15.37:59440) 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.940+0000 7f82495a9700 1 -- 172.21.15.37:0/695373313 learned_addr learned my addr 172.21.15.37:0/695373313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f8249daa700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 0x7f8244074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f824a5ab700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 0x7f82440757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 -- 172.21.15.37:0/695373313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 msgr2=0x7f8244074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 0x7f8244074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 -- 172.21.15.37:0/695373313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 msgr2=0x7f82440757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 0x7f82440757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 -- 172.21.15.37:0/695373313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441a75e0 con 0x7f8244103480 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f824a5ab700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 0x7f82440757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f82495a9700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f824000c9b0 tx=0x7f824000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240007b00 con 0x7f8244103480 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8240007ca0 con 0x7f8244103480 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240018660 con 0x7f8244103480 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441a78d0 con 0x7f8244103480 2024-04-25T21:13:34.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.941+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a7dc0 con 0x7f8244103480 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.942+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8244069730 con 0x7f8244103480 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.944+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f824000f450 con 0x7f8244103480 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.944+0000 7f8236ffd700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 0x7f82300a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.944+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f82400d0a50 con 0x7f8244103480 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.944+0000 7f8249daa700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 0x7f82300a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:34.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.945+0000 7f8249daa700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 0x7f82300a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82380098e0 tx=0x7f8238005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:34.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:34.948+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8240014030 con 0x7f8244103480 2024-04-25T21:13:35.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.205+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f824402e020 con 0x7f8244103480 2024-04-25T21:13:35.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.205+0000 7f8236ffd700 1 -- 172.21.15.37:0/695373313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f824009d1d0 con 0x7f8244103480 2024-04-25T21:13:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:13:35.207 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 msgr2=0x7f82300a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 0x7f82300a62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82380098e0 tx=0x7f8238005960 comp rx=0 tx=0).stop 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 msgr2=0x7f8244075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f824000c9b0 tx=0x7f824000ccc0 comp rx=0 tx=0).stop 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 shutdown_connections 2024-04-25T21:13:35.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f82300a3e30 0x7f82300a62f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8244108a70 0x7f82440757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8244102ac0 0x7f8244074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 --2- 172.21.15.37:0/695373313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8244103480 0x7f8244075280 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 >> 172.21.15.37:0/695373313 conn(0x7f8244079bb0 msgr2=0x7f82441018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 shutdown_connections 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:35.208+0000 7f824c00e700 1 -- 172.21.15.37:0/695373313 wait complete. 2024-04-25T21:13:35.209 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:13:35.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:35 smithi037 ceph-mon[29195]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-25T21:13:35.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:35 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/695373313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:35.971 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id d1f85f6c --release --await' 2024-04-25T21:13:35.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id d1f85f6c --release --await 2024-04-25T21:13:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:35 smithi177 ceph-mon[33043]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-25T21:13:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:35 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/695373313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:35 smithi098 ceph-mon[32569]: pgmap v156: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-25T21:13:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:35 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/695373313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:13:36.223 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:13:37.809 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:37 smithi037 ceph-mon[29195]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 -- 172.21.15.37:0/1550470863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 msgr2=0x7f2c8810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 --2- 172.21.15.37:0/1550470863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c8810e770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000b0a0 tx=0x7f2c8000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 -- 172.21.15.37:0/1550470863 shutdown_connections 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 --2- 172.21.15.37:0/1550470863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c8810e770 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 --2- 172.21.15.37:0/1550470863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c88106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 --2- 172.21.15.37:0/1550470863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c881021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 -- 172.21.15.37:0/1550470863 >> 172.21.15.37:0/1550470863 conn(0x7f2c880788f0 msgr2=0x7f2c88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 -- 172.21.15.37:0/1550470863 shutdown_connections 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.053+0000 7f2c900fd700 1 -- 172.21.15.37:0/1550470863 wait complete. 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 Processor -- start 2024-04-25T21:13:38.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 -- start start 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c88199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c8819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88111ae0 con 0x7f2c881026e0 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88111960 con 0x7f2c88106990 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c900fd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88111c60 con 0x7f2c88101d20 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.054+0000 7f2c8de99700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c88199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59456/0 (socket says 172.21.15.37:59456) 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 -- 172.21.15.37:0/3702928092 learned_addr learned my addr 172.21.15.37:0/3702928092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c8819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 -- 172.21.15.37:0/3702928092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 msgr2=0x7f2c88199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c88199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 -- 172.21.15.37:0/3702928092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 msgr2=0x7f2c8819d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c8819d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 -- 172.21.15.37:0/3702928092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8819e130 con 0x7f2c881026e0 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c8819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8de99700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c88199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9a0 tx=0x7f2c7800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c7effd700 1 -- 172.21.15.37:0/3702928092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800ca10 con 0x7f2c881026e0 2024-04-25T21:13:38.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c7effd700 1 -- 172.21.15.37:0/3702928092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7800cbb0 con 0x7f2c881026e0 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c7effd700 1 -- 172.21.15.37:0/3702928092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78018680 con 0x7f2c881026e0 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8819e420 con 0x7f2c881026e0 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.055+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c881a5ca0 con 0x7f2c881026e0 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.059+0000 7f2c7effd700 1 -- 172.21.15.37:0/3702928092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c78010430 con 0x7f2c881026e0 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.059+0000 7f2c9a986b80 1 Processor -- start 2024-04-25T21:13:38.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.059+0000 7f2c9a986b80 1 -- start start 2024-04-25T21:13:38.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.059+0000 7f2c7effd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 0x7f2c740a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.059+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7117d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f711714d90 con 0x55f7117c1f70 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f7116bba30 con 0x55f7117c1650 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f711710d50 con 0x55f7117d1280 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 0x7f2c740a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c7effd700 1 -- 172.21.15.37:0/3702928092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2c780d0940 con 0x7f2c881026e0 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59462/0 (socket says 172.21.15.37:59462) 2024-04-25T21:13:38.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 -- 172.21.15.37:0/1035611832 learned_addr learned my addr 172.21.15.37:0/1035611832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7117d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8de99700 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 -- 172.21.15.37:0/1035611832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 msgr2=0x55f7117d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7117d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 -- 172.21.15.37:0/1035611832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 msgr2=0x55f7117c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.060+0000 7f2c8d698700 1 -- 172.21.15.37:0/1035611832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71167f840 con 0x55f7117c1f70 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.061+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 0x7f2c740a6270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009fd0 tx=0x7f2c80009590 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.061+0000 7f2c8d698700 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2c780a5050 tx=0x7f2c7809d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e3c7b5010a16f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.061+0000 7f2c73fff700 1 -- 172.21.15.37:0/1035611832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780a5580 con 0x55f7117c1f70 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.061+0000 7f2c73fff700 1 -- 172.21.15.37:0/1035611832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c78085e50 con 0x55f7117c1f70 2024-04-25T21:13:38.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.061+0000 7f2c73fff700 1 -- 172.21.15.37:0/1035611832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78051be0 con 0x55f7117c1f70 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 -- 172.21.15.37:0/1035611832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 msgr2=0x55f7117d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2c780a5050 tx=0x7f2c7809d010 comp rx=0 tx=0).stop 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 -- 172.21.15.37:0/1035611832 shutdown_connections 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7117d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/1035611832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1f70 0x55f7117d0d40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 -- 172.21.15.37:0/1035611832 >> 172.21.15.37:0/1035611832 conn(0x55f7116ee990 msgr2=0x55f7117bdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 -- 172.21.15.37:0/1035611832 shutdown_connections 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.062+0000 7f2c9a986b80 1 -- 172.21.15.37:0/1035611832 wait complete. 2024-04-25T21:13:38.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 -- start start 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 Processor -- start 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117d1280 0x55f7117ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f7117d3d10 con 0x55f7117d1280 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55f7116ce6b0 con 0x55f7117c1650 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55f7117d3e90 con 0x55f7117c1f70 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.063+0000 7f2c9a986b80 10 client.0 _ll_register_callbacks cb 0x55f7116ada20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:48372/0 (socket says 172.21.15.37:48372) 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 -- 172.21.15.37:0/878050480 learned_addr learned my addr 172.21.15.37:0/878050480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117d1280 0x55f7117ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8de99700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 -- 172.21.15.37:0/878050480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 msgr2=0x55f7117d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 -- 172.21.15.37:0/878050480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117d1280 msgr2=0x55f7117ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117d1280 0x55f7117ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 -- 172.21.15.37:0/878050480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7116629d0 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c8d698700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c2d0 tx=0x7f2c78010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbde1c3596daf84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c717fa700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7809cd70 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c78085e50 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.064+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2c780a50f0 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c780a52f0 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 10 client.25549 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f7117d3d10 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c78085e50 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2c78010860 con 0x55f7117c1f70 2024-04-25T21:13:38.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 10 client.25549 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 10 client.25549 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 20 client.25549 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f7117c6bb0 0x55f7117c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 4 client.25549 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f711722580 con 0x55f7117c6bb0 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.065+0000 7f2c8de99700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f7117c6bb0 0x55f7117c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.066+0000 7f2c717fa700 10 client.25549 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:38.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.066+0000 7f2c8de99700 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f7117c6bb0 0x55f7117c6f90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe1d099c5068377 server_cookie=2a1aacb05214a95c in_seq=0 out_seq=0 2024-04-25T21:13:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.066+0000 7f2c717fa700 1 -- 172.21.15.37:0/878050480 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f711722580 con 0x55f7117c6bb0 2024-04-25T21:13:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.066+0000 7f2c717fa700 10 client.25549 handle_command_reply: tid=0 2024-04-25T21:13:38.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 client.25549 shutdown 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 msgr2=0x55f7117cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800c2d0 tx=0x7f2c78010c10 comp rx=0 tx=0).stop 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 shutdown_connections 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f7117c6bb0 0x55f7117c6f90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117d1280 0x55f7117ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1650 0x55f7117d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/878050480 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117cdd60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 >> 172.21.15.37:0/878050480 conn(0x55f7116ee990 msgr2=0x55f7117c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c717fa700 0 client.25549 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 shutdown_connections 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 1 -- 172.21.15.37:0/878050480 wait complete. 2024-04-25T21:13:38.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.067+0000 7f2c9a986b80 20 client.25549 trim_cache size 0 max 16384 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 Processor -- start 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 -- start start 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117d1280 0x55f7117e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881aafc0 con 0x55f7117c1650 2024-04-25T21:13:38.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881acb00 con 0x55f7117d1280 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881ab690 con 0x55f7117c1f70 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c8de99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c8de99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59468/0 (socket says 172.21.15.37:59468) 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c8de99700 1 -- 172.21.15.37:0/2192654663 learned_addr learned my addr 172.21.15.37:0/2192654663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c8d698700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.087+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117d1280 0x55f7117e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 -- 172.21.15.37:0/2192654663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117d1280 msgr2=0x55f7117e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117d1280 0x55f7117e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 -- 172.21.15.37:0/2192654663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 msgr2=0x55f7117d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 -- 172.21.15.37:0/2192654663 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7117b52d0 con 0x55f7117c1f70 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8de99700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c8d698700 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117e3570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c780ce6c0 tx=0x7f2c7809ae20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c24a1ddc16d7dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c737fe700 1 -- 172.21.15.37:0/2192654663 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780988d0 con 0x55f7117c1f70 2024-04-25T21:13:38.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c737fe700 1 -- 172.21.15.37:0/2192654663 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c78085e50 con 0x55f7117c1f70 2024-04-25T21:13:38.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.088+0000 7f2c737fe700 1 -- 172.21.15.37:0/2192654663 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c780bbad0 con 0x55f7117c1f70 2024-04-25T21:13:38.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:37 smithi177 ceph-mon[33043]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- 172.21.15.37:0/2192654663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 msgr2=0x55f7117e3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117e3570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2c780ce6c0 tx=0x7f2c7809ae20 comp rx=0 tx=0).stop 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- 172.21.15.37:0/2192654663 shutdown_connections 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117d1280 0x55f7117e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f7117d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/2192654663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117c1f70 0x55f7117e3570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- 172.21.15.37:0/2192654663 >> 172.21.15.37:0/2192654663 conn(0x55f7116ee990 msgr2=0x55f7117dc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- 172.21.15.37:0/2192654663 shutdown_connections 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- 172.21.15.37:0/2192654663 wait complete. 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 -- start start 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.089+0000 7f2c9a986b80 1 Processor -- start 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1f70 0x55f71183c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7118365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881ac430 con 0x55f7117c1650 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881abd60 con 0x55f7117c1f70 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2c881a8f60 con 0x55f7117d1280 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c9a986b80 10 client.0 _ll_register_callbacks cb 0x55f7116ada20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59478/0 (socket says 172.21.15.37:59478) 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 -- 172.21.15.37:0/3599290482 learned_addr learned my addr 172.21.15.37:0/3599290482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8e69a700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7118365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1f70 0x55f71183c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 -- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 msgr2=0x55f7118365a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7118365a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 -- 172.21.15.37:0/3599290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1f70 msgr2=0x55f71183c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1f70 0x55f71183c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.090+0000 7f2c8de99700 1 -- 172.21.15.37:0/3599290482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f711836ca0 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c8de99700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2c840295f0 tx=0x7f2c84029620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9c1f12d5097a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c84003680 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2c84003820 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c78085e50 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2c84003a20 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 10 client.16029 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f711843a50 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c78085e50 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2c84034480 con 0x55f7117c1650 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 10 client.16029 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.091+0000 7f2c9a986b80 20 client.16029 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:38.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f711844140 0x55f711846580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:38.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c9a986b80 4 client.16029 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d1f85f6c", "release": true, "await": true}] 2024-04-25T21:13:38.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d1f85f6c", "release": true, "await": true}) -- 0x55f71171c620 con 0x55f711844140 2024-04-25T21:13:38.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f711844140 0x55f711846580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:38.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c8d698700 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f711844140 0x55f711846580 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46d966cbbaa08c6 server_cookie=8efaba87a1ef9fdf in_seq=0 out_seq=0 2024-04-25T21:13:38.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.092+0000 7f2c717fa700 10 client.16029 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c717fa700 1 -- 172.21.15.37:0/3599290482 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f71171c620 con 0x55f711844140 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c717fa700 10 client.16029 handle_command_reply: tid=0 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 client.16029 shutdown 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 msgr2=0x55f71183c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2c840295f0 tx=0x7f2c84029620 comp rx=0 tx=0).stop 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 shutdown_connections 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55f711844140 0x55f711846580 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f7117d1280 0x55f7118365a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f7117c1650 0x55f71183c180 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 --2- 172.21.15.37:0/3599290482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55f7117c1f70 0x55f71183c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c717fa700 0 client.16029 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 >> 172.21.15.37:0/3599290482 conn(0x55f7116ee990 msgr2=0x55f7117c3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 shutdown_connections 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 1 -- 172.21.15.37:0/3599290482 wait complete. 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.096+0000 7f2c9a986b80 20 client.16029 trim_cache size 0 max 16384 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "d1f85f6c": { 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 16, 2024-04-25T21:13:38.098 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.8, 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:38.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 msgr2=0x7f2c740a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 0x7f2c740a6270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009fd0 tx=0x7f2c80009590 comp rx=0 tx=0).stop 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 msgr2=0x7f2c88199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:38.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800e9a0 tx=0x7f2c7800ecb0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 shutdown_connections 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2c740a3db0 0x7f2c740a6270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c88106990 0x7f2c8819d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c88101d20 0x7f2c88199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 --2- 172.21.15.37:0/3702928092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c881026e0 0x7f2c88199660 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 >> 172.21.15.37:0/3702928092 conn(0x7f2c880788f0 msgr2=0x7f2c8806b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 shutdown_connections 2024-04-25T21:13:38.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:13:38.097+0000 7f2c900fd700 1 -- 172.21.15.37:0/3702928092 wait complete. 2024-04-25T21:13:38.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:37 smithi098 ceph-mon[32569]: pgmap v157: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:38.800 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd1f85f6c', seconds elapsed: 37.1 2024-04-25T21:13:38.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:13:39.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:13:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:13:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:39 smithi037 ceph-mon[29195]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:13:39.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:13:39.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:13:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:39 smithi177 ceph-mon[33043]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:13:40.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:13:40.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:13:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:39 smithi098 ceph-mon[32569]: pgmap v158: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-25T21:13:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:13:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:13:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:13:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:41 smithi177 ceph-mon[33043]: pgmap v159: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-25T21:13:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:41 smithi098 ceph-mon[32569]: pgmap v159: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-25T21:13:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:41 smithi037 ceph-mon[29195]: pgmap v159: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.2 KiB/s wr, 15 op/s 2024-04-25T21:13:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:43 smithi177 ceph-mon[33043]: pgmap v160: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-04-25T21:13:44.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:43 smithi098 ceph-mon[32569]: pgmap v160: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-04-25T21:13:44.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:43 smithi037 ceph-mon[29195]: pgmap v160: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-04-25T21:13:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:45 smithi177 ceph-mon[33043]: pgmap v161: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:13:46.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:45 smithi098 ceph-mon[32569]: pgmap v161: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:13:46.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:45 smithi037 ceph-mon[29195]: pgmap v161: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:13:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 -- 172.21.15.37:0/466262738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074250 msgr2=0x7f6d1c074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 --2- 172.21.15.37:0/466262738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074250 0x7f6d1c074630 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009ca0 tx=0x7f6d0c02a950 comp rx=0 tx=0).stop 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 -- 172.21.15.37:0/466262738 shutdown_connections 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 --2- 172.21.15.37:0/466262738 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c10e840 0x7f6d1c110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 --2- 172.21.15.37:0/466262738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c074b70 0x7f6d1c10e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 --2- 172.21.15.37:0/466262738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074250 0x7f6d1c074630 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 -- 172.21.15.37:0/466262738 >> 172.21.15.37:0/466262738 conn(0x7f6d1c0793e0 msgr2=0x7f6d1c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 -- 172.21.15.37:0/466262738 shutdown_connections 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 -- 172.21.15.37:0/466262738 wait complete. 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.285+0000 7f6d221ed640 1 Processor -- start 2024-04-25T21:13:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 -- start start 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c074250 0x7f6d1c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c10e840 0x7f6d1c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c114440 con 0x7f6d1c074b70 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1142c0 con 0x7f6d1c10e840 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d221ed640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1145c0 con 0x7f6d1c074250 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43324/0 (socket says 172.21.15.37:43324) 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 -- 172.21.15.37:0/1756588240 learned_addr learned my addr 172.21.15.37:0/1756588240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c074250 0x7f6d1c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c10e840 0x7f6d1c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 -- 172.21.15.37:0/1756588240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c074250 msgr2=0x7f6d1c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c074250 0x7f6d1c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 -- 172.21.15.37:0/1756588240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c10e840 msgr2=0x7f6d1c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c10e840 0x7f6d1c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.286+0000 7f6d1affd640 1 -- 172.21.15.37:0/1756588240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c19af80 con 0x7f6d1c074b70 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d1affd640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000d950 tx=0x7f6d1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d18ff9640 1 -- 172.21.15.37:0/1756588240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10017070 con 0x7f6d1c074b70 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1c19b270 con 0x7f6d1c074b70 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d18ff9640 1 -- 172.21.15.37:0/1756588240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d10009c30 con 0x7f6d1c074b70 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d18ff9640 1 -- 172.21.15.37:0/1756588240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000f620 con 0x7f6d1c074b70 2024-04-25T21:13:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.287+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1c1a7580 con 0x7f6d1c074b70 2024-04-25T21:13:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.289+0000 7f6d18ff9640 1 -- 172.21.15.37:0/1756588240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d10010460 con 0x7f6d1c074b70 2024-04-25T21:13:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 Processor -- start 2024-04-25T21:13:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 -- start start 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f5ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560295ec3970 con 0x560295f5ef40 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560295e604a0 con 0x560295f5e620 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560295ebf930 con 0x560295f69800 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43330/0 (socket says 172.21.15.37:43330) 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/97273009 learned_addr learned my addr 172.21.15.37:0/97273009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d18ff9640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 0x7f6d040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1affd640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f5ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 0x7f6d040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/97273009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 msgr2=0x560295f6bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/97273009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 msgr2=0x560295f5ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f5ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/97273009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560295bd6210 con 0x560295f5ef40 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f5ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d1affd640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.290+0000 7f6d18ff9640 1 -- 172.21.15.37:0/1756588240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d100d0470 con 0x7f6d1c074b70 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02ae70 tx=0x7f6d0c02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4608567882a9f6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 0x7f6d040a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c100e60 tx=0x7f6d14005f90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d01ffb640 1 -- 172.21.15.37:0/97273009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c048070 con 0x560295f5ef40 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d01ffb640 1 -- 172.21.15.37:0/97273009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c003e20 con 0x560295f5ef40 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d01ffb640 1 -- 172.21.15.37:0/97273009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c034040 con 0x560295f5ef40 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d24fd8740 1 -- 172.21.15.37:0/97273009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 msgr2=0x560295f692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.291+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02ae70 tx=0x7f6d0c02f450 comp rx=0 tx=0).stop 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 -- 172.21.15.37:0/97273009 shutdown_connections 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f692c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/97273009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f5ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 -- 172.21.15.37:0/97273009 >> 172.21.15.37:0/97273009 conn(0x560295f5ab30 msgr2=0x560295f5af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 -- 172.21.15.37:0/97273009 shutdown_connections 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 -- 172.21.15.37:0/97273009 wait complete. 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 -- start start 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.292+0000 7f6d24fd8740 1 Processor -- start 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f66450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560295f6c3d0 con 0x560295f69800 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560295e5e0b0 con 0x560295f5e620 2024-04-25T21:13:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560295f6c550 con 0x560295f5ef40 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 10 client.0 _ll_register_callbacks cb 0x560295e4bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43342/0 (socket says 172.21.15.37:43342) 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 -- 172.21.15.37:0/3281996598 learned_addr learned my addr 172.21.15.37:0/3281996598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1affd640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f66450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 -- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 msgr2=0x560295f66450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f66450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 -- 172.21.15.37:0/3281996598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 msgr2=0x560295f69160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f69160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 -- 172.21.15.37:0/3281996598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560295ca2620 con 0x560295f69800 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400fb80 tx=0x7f6d140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dde72e23717dfbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6cfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14018070 con 0x560295f69800 2024-04-25T21:13:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d1401ddc0 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.293+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0c003e20 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d14029480 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560295f78360 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c003e20 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d14032520 con 0x560295f69800 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 20 client.16041 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295f797d0 0x560295f7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 4 client.16041 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560295ed1390 con 0x560295f797d0 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.294+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295f797d0 0x560295f7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.295+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295f797d0 0x560295f7bbc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c73b5e1efe2f1 server_cookie=8d43bb417e6901e1 in_seq=0 out_seq=0 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.295+0000 7f6cfaffd640 10 client.16041 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6cfaffd640 1 -- 172.21.15.37:0/3281996598 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560295ed1390 con 0x560295f797d0 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6cfaffd640 10 client.16041 handle_command_reply: tid=0 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 client.16041 shutdown 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 msgr2=0x560295f669b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400fb80 tx=0x7f6d140046c0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 shutdown_connections 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295f797d0 0x560295f7bbc0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f69800 0x560295f669b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f66450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6cfaffd640 0 client.16041 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/3281996598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f69160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 >> 172.21.15.37:0/3281996598 conn(0x560295f5ab30 msgr2=0x560295f608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 shutdown_connections 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 1 -- 172.21.15.37:0/3281996598 wait complete. 2024-04-25T21:13:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.296+0000 7f6d24fd8740 20 client.16041 trim_cache size 0 max 16384 2024-04-25T21:13:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.311+0000 7f6d24fd8740 1 Processor -- start 2024-04-25T21:13:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 -- start start 2024-04-25T21:13:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f59b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f69800 0x560295f814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1afd10 con 0x560295f5e620 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1b1850 con 0x560295f69800 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1b03e0 con 0x560295f5ef40 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43348/0 (socket says 172.21.15.37:43348) 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/1734155492 learned_addr learned my addr 172.21.15.37:0/1734155492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1affd640 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f59b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/1734155492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 msgr2=0x560295f59b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f59b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/1734155492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f69800 msgr2=0x560295f814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f69800 0x560295f814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f69800 0x560295f814d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 -- 172.21.15.37:0/1734155492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560295f66d20 con 0x560295f5e620 2024-04-25T21:13:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02fde0 tx=0x7f6d0c02fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d70f5aed05ff170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.312+0000 7f6d017fa640 1 -- 172.21.15.37:0/1734155492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c048040 con 0x560295f5e620 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d017fa640 1 -- 172.21.15.37:0/1734155492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c057070 con 0x560295f5e620 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d017fa640 1 -- 172.21.15.37:0/1734155492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c034040 con 0x560295f5e620 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 -- 172.21.15.37:0/1734155492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 msgr2=0x560295f75290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c02fde0 tx=0x7f6d0c02fd70 comp rx=0 tx=0).stop 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 -- 172.21.15.37:0/1734155492 shutdown_connections 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f69800 0x560295f814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f5ef40 0x560295f59b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/1734155492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5e620 0x560295f75290 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 -- 172.21.15.37:0/1734155492 >> 172.21.15.37:0/1734155492 conn(0x560295f5ab30 msgr2=0x560295f59420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 -- 172.21.15.37:0/1734155492 shutdown_connections 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.313+0000 7f6d24fd8740 1 -- 172.21.15.37:0/1734155492 wait complete. 2024-04-25T21:13:46.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 -- start start 2024-04-25T21:13:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 Processor -- start 2024-04-25T21:13:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1b1180 con 0x560295f5ef40 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1b0ab0 con 0x560295f5e620 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c1adcf0 con 0x560295f69800 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d24fd8740 10 client.0 _ll_register_callbacks cb 0x7f6d1400f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43356/0 (socket says 172.21.15.37:43356) 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 -- 172.21.15.37:0/4575007 learned_addr learned my addr 172.21.15.37:0/4575007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1bfff640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 -- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 msgr2=0x560295f6f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.314+0000 7f6d1affd640 1 -- 172.21.15.37:0/4575007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 msgr2=0x560295f71aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d1affd640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f71aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d1affd640 1 -- 172.21.15.37:0/4575007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560295f6fbf0 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f71aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d1affd640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d1009c370 tx=0x7f6d1009cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcb1fdb1a080a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10020040 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f6d1009b830 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0c057070 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6d100ce010 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560295fdfca0 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0c057070 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6d100b5b60 con 0x560295f5ef40 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 20 client.16047 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.315+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295fe0420 0x560295fe2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.316+0000 7f6d24fd8740 4 client.16047 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.316+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560295ecb200 con 0x560295fe0420 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.316+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295fe0420 0x560295fe2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.316+0000 7f6d1b7fe640 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295fe0420 0x560295fe2840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e085a341492f2c3a server_cookie=294c9b88a3bc7c9e in_seq=0 out_seq=0 2024-04-25T21:13:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.316+0000 7f6cfaffd640 10 client.16047 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.328+0000 7f6cfaffd640 1 -- 172.21.15.37:0/4575007 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560295ecb200 con 0x560295fe0420 2024-04-25T21:13:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.328+0000 7f6cfaffd640 10 client.16047 handle_command_reply: tid=0 2024-04-25T21:13:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 client.16047 shutdown 2024-04-25T21:13:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 msgr2=0x560295f6eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6d1009c370 tx=0x7f6d1009cb50 comp rx=0 tx=0).stop 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 shutdown_connections 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560295fe0420 0x560295fe2840 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560295f69800 0x560295f6f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560295f5ef40 0x560295f6eeb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 --2- 172.21.15.37:0/4575007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560295f5e620 0x560295f71aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6cfaffd640 0 client.16047 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 >> 172.21.15.37:0/4575007 conn(0x560295f5ab30 msgr2=0x560295f59cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 shutdown_connections 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 1 -- 172.21.15.37:0/4575007 wait complete. 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.329+0000 7f6d24fd8740 20 client.16047 trim_cache size 0 max 16384 2024-04-25T21:13:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:13:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:13:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:13:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:13:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 msgr2=0x7f6d040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 0x7f6d040a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c100e60 tx=0x7f6d14005f90 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 msgr2=0x7f6d1c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000d950 tx=0x7f6d1000de10 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 shutdown_connections 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d1c10e840 0x7f6d1c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d1c074b70 0x7f6d1c1a0b40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6d040a3dc0 0x7f6d040a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 --2- 172.21.15.37:0/1756588240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d1c074250 0x7f6d1c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 >> 172.21.15.37:0/1756588240 conn(0x7f6d1c0793e0 msgr2=0x7f6d1c0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 shutdown_connections 2024-04-25T21:13:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:46.330+0000 7f6d221ed640 1 -- 172.21.15.37:0/1756588240 wait complete. 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:64/600: open fd = 3 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:65/600: open fd = 3 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.170 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:66/600: open fd = 3 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:67/600: open fd = 3 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:68/600: open fd = 3 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:69/600: open fd = 3 2024-04-25T21:13:47.171 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:70/600: open fd = 3 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:71/600: open fd = 3 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:72/600: open fd = 3 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.172 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:73/600: open fd = 3 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:74/600: open fd = 3 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:75/600: open fd = 3 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:76/600: open fd = 3 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.173 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:77/600: open fd = 3 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:78/600: open fd = 3 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:79/600: open fd = 3 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.174 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:80/600: open fd = 3 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:81/600: open fd = 3 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:82/600: open fd = 3 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:83/600: open fd = 3 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.175 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:84/600: open fd = 3 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:85/600: open fd = 3 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:86/600: open fd = 3 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:87/600: open fd = 3 2024-04-25T21:13:47.176 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:88/600: open fd = 3 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:89/600: open fd = 3 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:90/600: open fd = 3 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:91/600: open fd = 3 2024-04-25T21:13:47.177 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:92/600: open fd = 3 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:93/600: open fd = 3 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:94/600: open fd = 3 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:95/600: open fd = 3 2024-04-25T21:13:47.178 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:96/600: open fd = 3 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:97/600: open fd = 3 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:98/600: open fd = 3 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.179 INFO:tasks.workunit.client.0.smithi037.stdout:99/600: open fd = 3 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:100/600: open fd = 3 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:101/600: open fd = 3 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:102/600: open fd = 3 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.180 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:103/600: open fd = 3 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:104/600: open fd = 3 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:105/600: open fd = 3 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:106/600: open fd = 3 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.181 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:107/600: open fd = 3 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:108/600: open fd = 3 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:109/600: open fd = 3 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.182 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:110/600: open fd = 3 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:111/600: open fd = 3 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:112/600: open fd = 3 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.183 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:113/600: open fd = 3 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:114/600: open fd = 3 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:115/600: open fd = 3 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:116/600: open fd = 3 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.184 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:117/600: open fd = 3 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:118/600: open fd = 3 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:119/600: open fd = 3 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:120/600: open fd = 3 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.185 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:121/600: open fd = 3 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:122/600: open fd = 3 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:123/600: open fd = 3 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:124/600: open fd = 3 2024-04-25T21:13:47.186 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:125/600: open fd = 3 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:126/600: open fd = 3 2024-04-25T21:13:47.187 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:13:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:47 smithi177 ceph-mon[33043]: pgmap v162: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:48.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:47 smithi098 ceph-mon[32569]: pgmap v162: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:47 smithi037 ceph-mon[29195]: pgmap v162: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:49 smithi177 ceph-mon[33043]: pgmap v163: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:49 smithi098 ceph-mon[32569]: pgmap v163: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:50.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:49 smithi037 ceph-mon[29195]: pgmap v163: 97 pgs: 97 active+clean; 4.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 11 op/s 2024-04-25T21:13:52.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:51 smithi037 ceph-mon[29195]: pgmap v164: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-25T21:13:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:51 smithi177 ceph-mon[33043]: pgmap v164: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-25T21:13:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:51 smithi098 ceph-mon[32569]: pgmap v164: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-25T21:13:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:53 smithi177 ceph-mon[33043]: pgmap v165: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:54.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:53 smithi098 ceph-mon[32569]: pgmap v165: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:53 smithi037 ceph-mon[29195]: pgmap v165: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:55 smithi177 ceph-mon[33043]: pgmap v166: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-04-25T21:13:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:55 smithi098 ceph-mon[32569]: pgmap v166: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-04-25T21:13:56.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:55 smithi037 ceph-mon[29195]: pgmap v166: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 21 op/s 2024-04-25T21:13:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:57 smithi177 ceph-mon[33043]: pgmap v167: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:58.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:57 smithi098 ceph-mon[32569]: pgmap v167: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:58.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:57 smithi037 ceph-mon[29195]: pgmap v167: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- 172.21.15.37:0/3330236941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c1010e0 msgr2=0x7f431c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 --2- 172.21.15.37:0/3330236941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c1010e0 0x7f431c1014c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f431802a410 comp rx=0 tx=0).stop 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- 172.21.15.37:0/3330236941 shutdown_connections 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 --2- 172.21.15.37:0/3330236941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1061b0 0x7f431c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 --2- 172.21.15.37:0/3330236941 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c101a90 0x7f431c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 --2- 172.21.15.37:0/3330236941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c1010e0 0x7f431c1014c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- 172.21.15.37:0/3330236941 >> 172.21.15.37:0/3330236941 conn(0x7f431c0771d0 msgr2=0x7f431c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- 172.21.15.37:0/3330236941 shutdown_connections 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- 172.21.15.37:0/3330236941 wait complete. 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 Processor -- start 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.431+0000 7f4322f30640 1 -- start start 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1010e0 0x7f431c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c1061b0 0x7f431c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c111110 con 0x7f431c19cf60 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f431c110f90 con 0x7f431c1010e0 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4322f30640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f431c111290 con 0x7f431c1061b0 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4320ccc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1010e0 0x7f431c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52574/0 (socket says 172.21.15.37:52574) 2024-04-25T21:13:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 -- 172.21.15.37:0/1027459947 learned_addr learned my addr 172.21.15.37:0/1027459947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4313fff640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c1061b0 0x7f431c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 -- 172.21.15.37:0/1027459947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c1061b0 msgr2=0x7f431c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c1061b0 0x7f431c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 -- 172.21.15.37:0/1027459947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1010e0 msgr2=0x7f431c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1010e0 0x7f431c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 -- 172.21.15.37:0/1027459947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431c19da50 con 0x7f431c19cf60 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f43214cd640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4314009fd0 tx=0x7f431400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.432+0000 7f4311ffb640 1 -- 172.21.15.37:0/1027459947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314017070 con 0x7f431c19cf60 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.433+0000 7f4311ffb640 1 -- 172.21.15.37:0/1027459947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314003e20 con 0x7f431c19cf60 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.433+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431c19dd40 con 0x7f431c19cf60 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.433+0000 7f4311ffb640 1 -- 172.21.15.37:0/1027459947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314010460 con 0x7f431c19cf60 2024-04-25T21:13:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.433+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f431c1a5530 con 0x7f431c19cf60 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.435+0000 7f4311ffb640 1 -- 172.21.15.37:0/1027459947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4314003970 con 0x7f431c19cf60 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.435+0000 7f4325c46740 1 Processor -- start 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.435+0000 7f4325c46740 1 -- start start 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.435+0000 7f4311ffb640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 0x7f43040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c3298b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c3298e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c328e1970 con 0x557c3297c4e0 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557c3287e4a0 con 0x557c3297ce00 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4325c46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c328dd930 con 0x557c3298bc10 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4311ffb640 1 -- 172.21.15.37:0/1027459947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f43140d0490 con 0x7f431c19cf60 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4313fff640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 0x7f43040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52590/0 (socket says 172.21.15.37:52590) 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 -- 172.21.15.37:0/2571830872 learned_addr learned my addr 172.21.15.37:0/2571830872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4313fff640 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c3298b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f43214cd640 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c3298e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 -- 172.21.15.37:0/2571830872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 msgr2=0x557c3298e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c3298e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 -- 172.21.15.37:0/2571830872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 msgr2=0x557c3298b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c3298b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 -- 172.21.15.37:0/2571830872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c325f4210 con 0x557c3297c4e0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4313fff640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 0x7f43040a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f431c107810 tx=0x7f430c00bb30 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.436+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f4318032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e3e46dbbf1cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4302ffd640 1 -- 172.21.15.37:0/2571830872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318040070 con 0x557c3297c4e0 2024-04-25T21:13:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4302ffd640 1 -- 172.21.15.37:0/2571830872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4318004030 con 0x557c3297c4e0 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4302ffd640 1 -- 172.21.15.37:0/2571830872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318035040 con 0x557c3297c4e0 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 -- 172.21.15.37:0/2571830872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 msgr2=0x557c3297c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f4318032bd0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 -- 172.21.15.37:0/2571830872 shutdown_connections 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c3298e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c3298b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 --2- 172.21.15.37:0/2571830872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297c4e0 0x557c3297c8c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 -- 172.21.15.37:0/2571830872 >> 172.21.15.37:0/2571830872 conn(0x557c32978b30 msgr2=0x557c32978f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 -- 172.21.15.37:0/2571830872 shutdown_connections 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.437+0000 7f4325c46740 1 -- 172.21.15.37:0/2571830872 wait complete. 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 -- start start 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 Processor -- start 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c3298b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c32988820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c3298e780 con 0x557c3298bc10 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557c3287c0b0 con 0x557c3297ce00 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557c3298e900 con 0x557c3297c4e0 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f4325c46740 10 client.0 _ll_register_callbacks cb 0x557c32869d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.438+0000 7f43214cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4320ccc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c3298b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52606/0 (socket says 172.21.15.37:52606) 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 -- 172.21.15.37:0/2866060523 learned_addr learned my addr 172.21.15.37:0/2866060523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4313fff640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c32988820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 -- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 msgr2=0x557c3298b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c3298b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 -- 172.21.15.37:0/2866060523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 msgr2=0x557c32988820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c32988820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 -- 172.21.15.37:0/2866060523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c326c0620 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4313fff640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c32988820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c3298b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f43214cd640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f431409c3f0 tx=0x7f4314003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b319490f35b4d1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f42fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314020040 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f4314098ad0 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4318004030 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43140a0810 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4325c46740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.439+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c3299a730 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4318004030 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f43140d0020 con 0x557c3298bc10 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 20 client.16059 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c3299bbf0 0x557c3299dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 4 client.16059 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c328ef390 con 0x557c3299bbf0 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.440+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c3299bbf0 0x557c3299dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.441+0000 7f4320ccc640 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c3299bbf0 0x557c3299dfe0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a85d77b882f408 server_cookie=98ef717b3c4b689e in_seq=0 out_seq=0 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.441+0000 7f42fbfff640 10 client.16059 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.441+0000 7f42fbfff640 1 -- 172.21.15.37:0/2866060523 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c328ef390 con 0x557c3299bbf0 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.441+0000 7f42fbfff640 10 client.16059 handle_command_reply: tid=0 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.441+0000 7f4325c46740 1 client.16059 shutdown 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 msgr2=0x557c32988d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f431409c3f0 tx=0x7f4314003c60 comp rx=0 tx=0).stop 2024-04-25T21:13:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 shutdown_connections 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c3299bbf0 0x557c3299dfe0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3298bc10 0x557c32988d60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297ce00 0x557c32988820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f42fbfff640 0 client.16059 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 --2- 172.21.15.37:0/2866060523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c3298b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 >> 172.21.15.37:0/2866060523 conn(0x557c32978b30 msgr2=0x557c3297e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 shutdown_connections 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 1 -- 172.21.15.37:0/2866060523 wait complete. 2024-04-25T21:13:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.442+0000 7f4325c46740 20 client.16059 trim_cache size 0 max 16384 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 Processor -- start 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 -- start start 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c32997680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3298bc10 0x557c3298b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ab7c0 con 0x557c3297ce00 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ad300 con 0x557c3298bc10 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4325c46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1abe90 con 0x557c3297c4e0 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4313fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4313fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52608/0 (socket says 172.21.15.37:52608) 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.457+0000 7f4313fff640 1 -- 172.21.15.37:0/1287009561 learned_addr learned my addr 172.21.15.37:0/1287009561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4320ccc640 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c32997680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f43214cd640 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3298bc10 0x557c3298b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 -- 172.21.15.37:0/1287009561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 msgr2=0x557c32997680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c32997680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 -- 172.21.15.37:0/1287009561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3298bc10 msgr2=0x557c3298b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3298bc10 0x557c3298b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 -- 172.21.15.37:0/1287009561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3297cc30 con 0x557c3297ce00 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f4313fff640 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f430c00b460 tx=0x7f430c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab899e1f030232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f43027fc640 1 -- 172.21.15.37:0/1287009561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c012970 con 0x557c3297ce00 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f43027fc640 1 -- 172.21.15.37:0/1287009561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c026070 con 0x557c3297ce00 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.458+0000 7f43027fc640 1 -- 172.21.15.37:0/1287009561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c022cf0 con 0x557c3297ce00 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- 172.21.15.37:0/1287009561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 msgr2=0x557c32983d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f430c00b460 tx=0x7f430c01e000 comp rx=0 tx=0).stop 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- 172.21.15.37:0/1287009561 shutdown_connections 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3298bc10 0x557c3298b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32983d80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 --2- 172.21.15.37:0/1287009561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3297c4e0 0x557c32997680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- 172.21.15.37:0/1287009561 >> 172.21.15.37:0/1287009561 conn(0x557c32978b30 msgr2=0x557c32983670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- 172.21.15.37:0/1287009561 shutdown_connections 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- 172.21.15.37:0/1287009561 wait complete. 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 -- start start 2024-04-25T21:13:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.459+0000 7f4325c46740 1 Processor -- start 2024-04-25T21:13:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297c4e0 0x557c32993e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c32985690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1acc30 con 0x557c3297ce00 2024-04-25T21:13:59.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1ac560 con 0x557c3297c4e0 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f431c1a97e0 con 0x557c3298bc10 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 10 client.0 _ll_register_callbacks cb 0x7f431409bfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52624/0 (socket says 172.21.15.37:52624) 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 -- 172.21.15.37:0/3593132066 learned_addr learned my addr 172.21.15.37:0/3593132066 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4320ccc640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297c4e0 0x557c32993e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f43214cd640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c32985690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 -- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 msgr2=0x557c32985690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c32985690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 -- 172.21.15.37:0/3593132066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297c4e0 msgr2=0x557c32993e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297c4e0 0x557c32993e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 -- 172.21.15.37:0/3593132066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c32985dd0 con 0x557c3297ce00 2024-04-25T21:13:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4313fff640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f430c0127c0 tx=0x7f430c014340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad4cf12e7936f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f42fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00b490 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f430c012d80 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.460+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c026070 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f430c041050 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557c32a02080 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c026070 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f430c02b6d0 con 0x557c3297ce00 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 20 client.16065 populate_metadata read hostname 'smithi037' 2024-04-25T21:13:59.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c32a02770 0x557c32a04b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:13:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 4 client.16065 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:13:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c329890f0 con 0x557c32a02770 2024-04-25T21:13:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.461+0000 7f4320ccc640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c32a02770 0x557c32a04b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:13:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.462+0000 7f42fbfff640 10 client.16065 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:59.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.462+0000 7f4320ccc640 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c32a02770 0x557c32a04b90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4c5334faa4650b0 server_cookie=b890929ad4d27c50 in_seq=0 out_seq=0 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f42fbfff640 1 -- 172.21.15.37:0/3593132066 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c329890f0 con 0x557c32a02770 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f42fbfff640 10 client.16065 handle_command_reply: tid=0 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 client.16065 shutdown 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 msgr2=0x557c32985150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f430c0127c0 tx=0x7f430c014340 comp rx=0 tx=0).stop 2024-04-25T21:13:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 shutdown_connections 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557c32a02770 0x557c32a04b90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557c3298bc10 0x557c32985690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c3297ce00 0x557c32985150 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f42fbfff640 0 client.16065 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 --2- 172.21.15.37:0/3593132066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557c3297c4e0 0x557c32993e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.475+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 >> 172.21.15.37:0/3593132066 conn(0x557c32978b30 msgr2=0x557c32983f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 shutdown_connections 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4325c46740 1 -- 172.21.15.37:0/3593132066 wait complete. 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4325c46740 20 client.16065 trim_cache size 0 max 16384 2024-04-25T21:13:59.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:13:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:13:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:13:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:13:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 msgr2=0x7f43040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 0x7f43040a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f431c107810 tx=0x7f430c00bb30 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 msgr2=0x7f431c19d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4314009fd0 tx=0x7f431400b260 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 shutdown_connections 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f431c19cf60 0x7f431c19d410 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f43040a3e40 0x7f43040a6300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f431c1061b0 0x7f431c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 --2- 172.21.15.37:0/1027459947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f431c1010e0 0x7f431c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 >> 172.21.15.37:0/1027459947 conn(0x7f431c0771d0 msgr2=0x7f431c06aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 shutdown_connections 2024-04-25T21:13:59.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:13:59.476+0000 7f4322f30640 1 -- 172.21.15.37:0/1027459947 wait complete. 2024-04-25T21:14:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:13:59 smithi177 ceph-mon[33043]: pgmap v168: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:14:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:13:59 smithi098 ceph-mon[32569]: pgmap v168: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:14:00.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:13:59 smithi037 ceph-mon[29195]: pgmap v168: 97 pgs: 97 active+clean; 5.0 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-25T21:14:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:01 smithi177 ceph-mon[33043]: pgmap v169: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-25T21:14:02.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:01 smithi098 ceph-mon[32569]: pgmap v169: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-25T21:14:02.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:01 smithi037 ceph-mon[29195]: pgmap v169: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-25T21:14:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:03 smithi177 ceph-mon[33043]: pgmap v170: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T21:14:04.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:03 smithi098 ceph-mon[32569]: pgmap v170: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T21:14:04.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:03 smithi037 ceph-mon[29195]: pgmap v170: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-25T21:14:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:05 smithi177 ceph-mon[33043]: pgmap v171: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-25T21:14:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:05 smithi098 ceph-mon[32569]: pgmap v171: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-25T21:14:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:05 smithi037 ceph-mon[29195]: pgmap v171: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-25T21:14:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:07 smithi177 ceph-mon[33043]: pgmap v172: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:07 smithi098 ceph-mon[32569]: pgmap v172: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:07 smithi037 ceph-mon[29195]: pgmap v172: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:09 smithi177 ceph-mon[33043]: pgmap v173: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:09 smithi098 ceph-mon[32569]: pgmap v173: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:10.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:09 smithi037 ceph-mon[29195]: pgmap v173: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-25T21:14:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:11 smithi177 ceph-mon[33043]: pgmap v174: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-25T21:14:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:11 smithi098 ceph-mon[32569]: pgmap v174: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-25T21:14:12.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:11 smithi037 ceph-mon[29195]: pgmap v174: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 75 KiB/s wr, 38 op/s 2024-04-25T21:14:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/2946879624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff40742f0 msgr2=0x7f2ff40746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/2946879624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff40742f0 0x7f2ff40746d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009900 tx=0x7f2ff002a480 comp rx=0 tx=0).stop 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/2946879624 shutdown_connections 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/2946879624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff4112f60 0x7f2ff4115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/2946879624 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4074c10 0x7f2ff4112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/2946879624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff40742f0 0x7f2ff40746d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/2946879624 >> 172.21.15.37:0/2946879624 conn(0x7f2ff40fda20 msgr2=0x7f2ff40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/2946879624 shutdown_connections 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.576+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/2946879624 wait complete. 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 Processor -- start 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 -- start start 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 0x7f2ff41a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 0x7f2ff419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41187e0 con 0x7f2ff4074c10 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4118660 con 0x7f2ff40742f0 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.577+0000 7f2ffbd2d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4118960 con 0x7f2ff4112f60 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45436/0 (socket says 172.21.15.37:45436) 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 -- 172.21.15.37:0/3057755587 learned_addr learned my addr 172.21.15.37:0/3057755587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 0x7f2ff419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 0x7f2ff41a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 -- 172.21.15.37:0/3057755587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 msgr2=0x7f2ff419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 0x7f2ff419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 -- 172.21.15.37:0/3057755587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 msgr2=0x7f2ff41a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 0x7f2ff41a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 -- 172.21.15.37:0/3057755587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff419f1c0 con 0x7f2ff4074c10 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 0x7f2ff419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 0x7f2ff41a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c890 tx=0x7f2fe400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2feaffd640 1 -- 172.21.15.37:0/3057755587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4004220 con 0x7f2ff4074c10 2024-04-25T21:14:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2feaffd640 1 -- 172.21.15.37:0/3057755587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe40043c0 con 0x7f2ff4074c10 2024-04-25T21:14:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff419f4b0 con 0x7f2ff4074c10 2024-04-25T21:14:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2feaffd640 1 -- 172.21.15.37:0/3057755587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4004a00 con 0x7f2ff4074c10 2024-04-25T21:14:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.578+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff41ab750 con 0x7f2ff4074c10 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.581+0000 7f2feaffd640 1 -- 172.21.15.37:0/3057755587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fe40079b0 con 0x7f2ff4074c10 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.581+0000 7f2ffea3d740 1 Processor -- start 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.581+0000 7f2ffea3d740 1 -- start start 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 0x560ce74612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce7463bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560ce73bb970 con 0x560ce7456620 2024-04-25T21:14:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560ce73584a0 con 0x560ce7456f40 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ce73b7930 con 0x560ce7461800 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2feaffd640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 0x7f2fe00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45442/0 (socket says 172.21.15.37:45442) 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/1289263331 learned_addr learned my addr 172.21.15.37:0/1289263331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce7463bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 0x560ce74612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 0x7f2fe00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2feaffd640 1 -- 172.21.15.37:0/3057755587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2fe40d05d0 con 0x7f2ff4074c10 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/1289263331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 msgr2=0x560ce7463bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce7463bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/1289263331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 msgr2=0x560ce74612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 0x560ce74612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/1289263331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce70ce210 con 0x560ce7456620 2024-04-25T21:14:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.582+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 0x560ce74612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:14:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.583+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fec013df0 tx=0x7f2fec010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390fa8260192cad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.583+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 0x7f2fe00a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005cc0 tx=0x7f2fec01b000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.583+0000 7f2fdffff640 1 -- 172.21.15.37:0/1289263331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec004d20 con 0x560ce7456620 2024-04-25T21:14:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.583+0000 7f2fdffff640 1 -- 172.21.15.37:0/1289263331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fec007bf0 con 0x560ce7456620 2024-04-25T21:14:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.583+0000 7f2fdffff640 1 -- 172.21.15.37:0/1289263331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec004290 con 0x560ce7456620 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- 172.21.15.37:0/1289263331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 msgr2=0x560ce7456a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2fec013df0 tx=0x7f2fec010740 comp rx=0 tx=0).stop 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- 172.21.15.37:0/1289263331 shutdown_connections 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce7463bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456f40 0x560ce74612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/1289263331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce7456a00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- 172.21.15.37:0/1289263331 >> 172.21.15.37:0/1289263331 conn(0x560ce7452b30 msgr2=0x560ce7452f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- 172.21.15.37:0/1289263331 shutdown_connections 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- 172.21.15.37:0/1289263331 wait complete. 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 -- start start 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.584+0000 7f2ffea3d740 1 Processor -- start 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce74611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce745ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560ce74643d0 con 0x560ce7456f40 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560ce73560b0 con 0x560ce7456620 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560ce7464550 con 0x560ce7461800 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffea3d740 10 client.0 _ll_register_callbacks cb 0x560ce7343d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45456/0 (socket says 172.21.15.37:45456) 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 -- 172.21.15.37:0/2845973148 learned_addr learned my addr 172.21.15.37:0/2845973148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce74611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce745ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 -- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 msgr2=0x560ce745ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce745ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 -- 172.21.15.37:0/2845973148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 msgr2=0x560ce74611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce74611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 -- 172.21.15.37:0/2845973148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce719a620 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fe409c530 tx=0x7f2fe406af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7211d4ba9904aaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.585+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401d070 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2fe409b9f0 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ce73c9390 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fe409bf00 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ce7470340 con 0x560ce7456f40 2024-04-25T21:14:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ce73c9390 con 0x560ce7456f40 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fe4085ae0 con 0x560ce7456f40 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 20 client.16077 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce7471800 0x560ce7473bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 4 client.16077 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce73c3200 con 0x560ce7471800 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.586+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce7471800 0x560ce7473bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.587+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce7471800 0x560ce7473bf0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2780032b6ca0d8e4 server_cookie=902d62a7669c9a5a in_seq=0 out_seq=0 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.587+0000 7f2fdd7fa640 10 client.16077 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2845973148 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ce73c3200 con 0x560ce7471800 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2fdd7fa640 10 client.16077 handle_command_reply: tid=0 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 client.16077 shutdown 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 msgr2=0x560ce745e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fe409c530 tx=0x7f2fe406af50 comp rx=0 tx=0).stop 2024-04-25T21:14:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 shutdown_connections 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce7471800 0x560ce7473bf0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7461800 0x560ce745ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456f40 0x560ce745e4a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2845973148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce74611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2fdd7fa640 0 client.16077 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 >> 172.21.15.37:0/2845973148 conn(0x560ce7452b30 msgr2=0x560ce7458860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 shutdown_connections 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2845973148 wait complete. 2024-04-25T21:14:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.588+0000 7f2ffea3d740 20 client.16077 trim_cache size 0 max 16384 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 Processor -- start 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 -- start start 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce7460ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7461800 0x560ce7479500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b3f90 con 0x560ce7456620 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b5ad0 con 0x560ce7461800 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b4660 con 0x560ce7456f40 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ff9ac9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ff9ac9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45468/0 (socket says 172.21.15.37:45468) 2024-04-25T21:14:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ff9ac9640 1 -- 172.21.15.37:0/828261085 learned_addr learned my addr 172.21.15.37:0/828261085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.603+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce7460ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 -- 172.21.15.37:0/828261085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 msgr2=0x560ce7460ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce7460ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 -- 172.21.15.37:0/828261085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7461800 msgr2=0x560ce7479500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7461800 0x560ce7479500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 -- 172.21.15.37:0/828261085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce745edb0 con 0x560ce7456620 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2ff002a9a0 tx=0x7f2ff002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58677df703612fad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2fdf7fe640 1 -- 172.21.15.37:0/828261085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0035040 con 0x560ce7456620 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2fdf7fe640 1 -- 172.21.15.37:0/828261085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff00054d0 con 0x560ce7456620 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.604+0000 7f2fdf7fe640 1 -- 172.21.15.37:0/828261085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0045850 con 0x560ce7456620 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- 172.21.15.37:0/828261085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 msgr2=0x560ce746d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2ff002a9a0 tx=0x7f2ff002a480 comp rx=0 tx=0).stop 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- 172.21.15.37:0/828261085 shutdown_connections 2024-04-25T21:14:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7461800 0x560ce7479500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce7460ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/828261085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7456620 0x560ce746d290 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- 172.21.15.37:0/828261085 >> 172.21.15.37:0/828261085 conn(0x560ce7452b30 msgr2=0x560ce74599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- 172.21.15.37:0/828261085 shutdown_connections 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- 172.21.15.37:0/828261085 wait complete. 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- start start 2024-04-25T21:14:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 Processor -- start 2024-04-25T21:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce745b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce745b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b5400 con 0x560ce7461800 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b4d30 con 0x560ce7456620 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.605+0000 7f2ffea3d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff41b1f70 con 0x560ce7456f40 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffea3d740 10 client.0 _ll_register_callbacks cb 0x7f2fe400a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45480/0 (socket says 172.21.15.37:45480) 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/2512907228 learned_addr learned my addr 172.21.15.37:0/2512907228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce745b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ff92c8640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce745b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 msgr2=0x560ce745b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce745b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/2512907228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 msgr2=0x560ce745b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce745b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 -- 172.21.15.37:0/2512907228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce74d6fc0 con 0x560ce7461800 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffa2ca640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2fec013df0 tx=0x7f2fec00adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad50f80e887341e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec013250 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f2fec0050b0 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.606+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff00054d0 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fec013950 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 10 client.16083 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ce74d82d0 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff00054d0 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fec01c030 con 0x560ce7461800 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 20 client.16083 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce74d87c0 0x560ce74dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 4 client.16083 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ce733f5f0 con 0x560ce74d87c0 2024-04-25T21:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.607+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce74d87c0 0x560ce74dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.608+0000 7f2fdd7fa640 10 client.16083 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.608+0000 7f2ff9ac9640 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce74d87c0 0x560ce74dabb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7551d4074928e141 server_cookie=9f17487ee6d932a in_seq=0 out_seq=0 2024-04-25T21:14:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.621+0000 7f2fdd7fa640 1 -- 172.21.15.37:0/2512907228 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ce733f5f0 con 0x560ce74d87c0 2024-04-25T21:14:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.621+0000 7f2fdd7fa640 10 client.16083 handle_command_reply: tid=0 2024-04-25T21:14:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.621+0000 7f2ffea3d740 1 client.16083 shutdown 2024-04-25T21:14:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 msgr2=0x560ce745be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2fec013df0 tx=0x7f2fec00adc0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 shutdown_connections 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560ce74d87c0 0x560ce74dabb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560ce7461800 0x560ce745be40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560ce7456f40 0x560ce745b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2fdd7fa640 0 client.16083 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 --2- 172.21.15.37:0/2512907228 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560ce7456620 0x560ce745b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 >> 172.21.15.37:0/2512907228 conn(0x560ce7452b30 msgr2=0x560ce7457fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 shutdown_connections 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 1 -- 172.21.15.37:0/2512907228 wait complete. 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.622+0000 7f2ffea3d740 20 client.16083 trim_cache size 0 max 16384 2024-04-25T21:14:12.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:14:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:14:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:14:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:14:12.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 msgr2=0x7f2fe00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 0x7f2fe00a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005cc0 tx=0x7f2fec01b000 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 msgr2=0x7f2ff41a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c890 tx=0x7f2fe400cd50 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 shutdown_connections 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2fe00a3dc0 0x7f2fe00a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff4112f60 0x7f2ff419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff4074c10 0x7f2ff41a4dd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 --2- 172.21.15.37:0/3057755587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ff40742f0 0x7f2ff41a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 >> 172.21.15.37:0/3057755587 conn(0x7f2ff40fda20 msgr2=0x7f2ff40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 shutdown_connections 2024-04-25T21:14:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:12.623+0000 7f2ffbd2d640 1 -- 172.21.15.37:0/3057755587 wait complete. 2024-04-25T21:14:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:13 smithi177 ceph-mon[33043]: pgmap v175: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-04-25T21:14:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:13 smithi098 ceph-mon[32569]: pgmap v175: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-04-25T21:14:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:13 smithi037 ceph-mon[29195]: pgmap v175: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 60 KiB/s wr, 31 op/s 2024-04-25T21:14:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:15 smithi177 ceph-mon[33043]: pgmap v176: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 40 op/s 2024-04-25T21:14:16.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:15 smithi098 ceph-mon[32569]: pgmap v176: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 40 op/s 2024-04-25T21:14:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:15 smithi037 ceph-mon[29195]: pgmap v176: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 83 KiB/s wr, 40 op/s 2024-04-25T21:14:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:17 smithi177 ceph-mon[33043]: pgmap v177: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:17 smithi098 ceph-mon[32569]: pgmap v177: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:18.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:17 smithi037 ceph-mon[29195]: pgmap v177: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:19 smithi177 ceph-mon[33043]: pgmap v178: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:19 smithi098 ceph-mon[32569]: pgmap v178: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:20.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:19 smithi037 ceph-mon[29195]: pgmap v178: 97 pgs: 97 active+clean; 1.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-25T21:14:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:21 smithi177 ceph-mon[33043]: pgmap v179: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:14:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:21 smithi098 ceph-mon[32569]: pgmap v179: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:14:22.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:21 smithi037 ceph-mon[29195]: pgmap v179: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:14:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:23 smithi177 ceph-mon[33043]: pgmap v180: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:23 smithi098 ceph-mon[32569]: pgmap v180: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:24.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:23 smithi037 ceph-mon[29195]: pgmap v180: 97 pgs: 97 active+clean; 1.9 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 -- 172.21.15.37:0/3267862995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc1010e0 msgr2=0x7fb5bc1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/3267862995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc1014c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009ca0 tx=0x7fb5b802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 -- 172.21.15.37:0/3267862995 shutdown_connections 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/3267862995 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 0x7fb5bc10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/3267862995 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc101a90 0x7fb5bc105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/3267862995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc1014c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 -- 172.21.15.37:0/3267862995 >> 172.21.15.37:0/3267862995 conn(0x7fb5bc0771d0 msgr2=0x7fb5bc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 -- 172.21.15.37:0/3267862995 shutdown_connections 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.722+0000 7fb5c38bb640 1 -- 172.21.15.37:0/3267862995 wait complete. 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 Processor -- start 2024-04-25T21:14:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 -- start start 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 0x7fb5bc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc111170 con 0x7fb5bc19ce90 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc110ff0 con 0x7fb5bc1010e0 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c38bb640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1112f0 con 0x7fb5bc1061b0 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c1e58640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c1e58640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39524/0 (socket says 172.21.15.37:39524) 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.723+0000 7fb5c1e58640 1 -- 172.21.15.37:0/2585097805 learned_addr learned my addr 172.21.15.37:0/2585097805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1657640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 0x7fb5bc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 -- 172.21.15.37:0/2585097805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 msgr2=0x7fb5bc198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 0x7fb5bc198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 -- 172.21.15.37:0/2585097805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc1010e0 msgr2=0x7fb5bc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 -- 172.21.15.37:0/2585097805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc19d980 con 0x7fb5bc19ce90 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c1e58640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c970 tx=0x7fb5b400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5b27fc640 1 -- 172.21.15.37:0/2585097805 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b400ae60 con 0x7fb5bc19ce90 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5b27fc640 1 -- 172.21.15.37:0/2585097805 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b40054d0 con 0x7fb5bc19ce90 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc19dc70 con 0x7fb5bc19ce90 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5b27fc640 1 -- 172.21.15.37:0/2585097805 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4004d60 con 0x7fb5bc19ce90 2024-04-25T21:14:25.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.724+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a54b0 con 0x7fb5bc19ce90 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5b27fc640 1 -- 172.21.15.37:0/2585097805 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5b4005020 con 0x7fb5bc19ce90 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 Processor -- start 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 -- start start 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617e20 0x5622e76266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7629000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5622e757c970 con 0x5622e7617500 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5622e75194a0 con 0x5622e7617e20 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5622e7578930 con 0x5622e7626c10 2024-04-25T21:14:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5b27fc640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 0x7fb5a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c0e56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.727+0000 7fb5c1e58640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 0x7fb5a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39540/0 (socket says 172.21.15.37:39540) 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 -- 172.21.15.37:0/1298402406 learned_addr learned my addr 172.21.15.37:0/1298402406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c1657640 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617e20 0x5622e76266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5b27fc640 1 -- 172.21.15.37:0/2585097805 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb5b40d0380 con 0x7fb5bc19ce90 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7629000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 -- 172.21.15.37:0/1298402406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 msgr2=0x5622e7629000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7629000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 -- 172.21.15.37:0/1298402406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617e20 msgr2=0x5622e76266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617e20 0x5622e76266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 -- 172.21.15.37:0/1298402406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e728f210 con 0x5622e7617500 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c1e58640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 0x7fb5a80a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40cdbb0 tx=0x7fb5b406dd90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac013580 tx=0x7fb5ac00a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6e0ab091486c7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5a77fe640 1 -- 172.21.15.37:0/1298402406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac019030 con 0x5622e7617500 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5a77fe640 1 -- 172.21.15.37:0/1298402406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac0110d0 con 0x5622e7617500 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.728+0000 7fb5a77fe640 1 -- 172.21.15.37:0/1298402406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00f800 con 0x5622e7617500 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- 172.21.15.37:0/1298402406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 msgr2=0x5622e76178e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac013580 tx=0x7fb5ac00a7c0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- 172.21.15.37:0/1298402406 shutdown_connections 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7629000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617e20 0x5622e76266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/1298402406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e76178e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- 172.21.15.37:0/1298402406 >> 172.21.15.37:0/1298402406 conn(0x5622e7613b30 msgr2=0x5622e7613f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- 172.21.15.37:0/1298402406 shutdown_connections 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- 172.21.15.37:0/1298402406 wait complete. 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 -- start start 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.729+0000 7fb5c65ce740 1 Processor -- start 2024-04-25T21:14:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 0x5622e7626550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7623d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5622e7629840 con 0x5622e7617e20 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5622e75170b0 con 0x5622e7617500 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5622e76299c0 con 0x5622e7626c10 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c65ce740 10 client.0 _ll_register_callbacks cb 0x5622e7504d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39552/0 (socket says 172.21.15.37:39552) 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 -- 172.21.15.37:0/3845723198 learned_addr learned my addr 172.21.15.37:0/3845723198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 0x5622e7626550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7623d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 -- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 msgr2=0x5622e7623d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7623d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 -- 172.21.15.37:0/3845723198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 msgr2=0x5622e7626550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.730+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 0x5622e7626550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c1657640 1 -- 172.21.15.37:0/3845723198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e735b620 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 0x5622e7626550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009ca0 tx=0x7fb5b8031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29c53be200e39ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8003970 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb5b8003b10 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac0110d0 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5b8003d10 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c65ce740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e7635800 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac0110d0 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.731+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5b8004650 con 0x5622e7617e20 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 20 client.16095 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7636d10 0x5622e7639100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 4 client.16095 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e758a390 con 0x5622e7636d10 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7636d10 0x5622e7639100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7636d10 0x5622e7639100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b34b8b237dfce1 server_cookie=2d0e59bb5e119437 in_seq=0 out_seq=0 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.732+0000 7fb5a4ff9640 10 client.16095 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3845723198 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622e758a390 con 0x5622e7636d10 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5a4ff9640 10 client.16095 handle_command_reply: tid=0 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 client.16095 shutdown 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 msgr2=0x5622e7623840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009ca0 tx=0x7fb5b8031680 comp rx=0 tx=0).stop 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 shutdown_connections 2024-04-25T21:14:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7636d10 0x5622e7639100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7626c10 0x5622e7623d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7623840 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3845723198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7617500 0x5622e7626550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5a4ff9640 0 client.16095 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 >> 172.21.15.37:0/3845723198 conn(0x5622e7613b30 msgr2=0x5622e7619940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 shutdown_connections 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3845723198 wait complete. 2024-04-25T21:14:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.733+0000 7fb5c65ce740 20 client.16095 trim_cache size 0 max 16384 2024-04-25T21:14:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.748+0000 7fb5c65ce740 1 Processor -- start 2024-04-25T21:14:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.748+0000 7fb5c65ce740 1 -- start start 2024-04-25T21:14:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e7632770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617500 0x5622e761ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1ab750 con 0x5622e7617e20 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1ad290 con 0x5622e7613b30 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1abe20 con 0x5622e7617500 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39562/0 (socket says 172.21.15.37:39562) 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 -- 172.21.15.37:0/65368759 learned_addr learned my addr 172.21.15.37:0/65368759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e7632770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c1657640 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617500 0x5622e761ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 -- 172.21.15.37:0/65368759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617500 msgr2=0x5622e761ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617500 0x5622e761ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 -- 172.21.15.37:0/65368759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 msgr2=0x5622e7632770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e7632770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 -- 172.21.15.37:0/65368759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e7614cf0 con 0x5622e7617e20 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac015010 tx=0x7fb5ac015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22ba227b499c974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5a6ffd640 1 -- 172.21.15.37:0/65368759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac031070 con 0x5622e7617e20 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5a6ffd640 1 -- 172.21.15.37:0/65368759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac01a040 con 0x5622e7617e20 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.749+0000 7fb5a6ffd640 1 -- 172.21.15.37:0/65368759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac019040 con 0x5622e7617e20 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- 172.21.15.37:0/65368759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 msgr2=0x5622e7626080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac015010 tx=0x7fb5ac015040 comp rx=0 tx=0).stop 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- 172.21.15.37:0/65368759 shutdown_connections 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617e20 0x5622e7626080 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617500 0x5622e761ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/65368759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e7632770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- 172.21.15.37:0/65368759 >> 172.21.15.37:0/65368759 conn(0x5622e7623ee0 msgr2=0x5622e761e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- 172.21.15.37:0/65368759 shutdown_connections 2024-04-25T21:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- 172.21.15.37:0/65368759 wait complete. 2024-04-25T21:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.750+0000 7fb5c65ce740 1 -- start start 2024-04-25T21:14:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 Processor -- start 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e762eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 0x5622e762a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1acbc0 con 0x5622e7617500 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1ac4f0 con 0x5622e7613b30 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc1a9780 con 0x5622e7617e20 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c65ce740 10 client.0 _ll_register_callbacks cb 0x7fb5b802f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39568/0 (socket says 172.21.15.37:39568) 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 -- 172.21.15.37:0/3542556149 learned_addr learned my addr 172.21.15.37:0/3542556149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 0x5622e762a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 -- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 msgr2=0x5622e762a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 0x5622e762a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 -- 172.21.15.37:0/3542556149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 msgr2=0x5622e762eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e762eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c1657640 1 -- 172.21.15.37:0/3542556149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e762b0d0 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.751+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 0x5622e762a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c1657640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000f80 tx=0x7fb5b802ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72a1c08fe888544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8009800 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7fb5b8042040 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5ac01a040 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb5b802f8a0 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e769d220 con 0x5622e7617500 2024-04-25T21:14:25.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac01a040 con 0x5622e7617500 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb5b8030980 con 0x5622e7617500 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 20 client.16101 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7626c10 0x5622e769f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 4 client.16101 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.752+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622e7584200 con 0x5622e7626c10 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.753+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7626c10 0x5622e769f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.753+0000 7fb5c0e56640 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7626c10 0x5622e769f9b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff939fd8eb7c1805 server_cookie=3a84f0fa71be973c in_seq=0 out_seq=0 2024-04-25T21:14:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.753+0000 7fb5a4ff9640 10 client.16101 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5a4ff9640 1 -- 172.21.15.37:0/3542556149 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622e7584200 con 0x5622e7626c10 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5a4ff9640 10 client.16101 handle_command_reply: tid=0 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 client.16101 shutdown 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 msgr2=0x5622e762a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000f80 tx=0x7fb5b802ffd0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 shutdown_connections 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e7626c10 0x5622e769f9b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e7617e20 0x5622e762a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5a4ff9640 0 client.16101 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e7617500 0x5622e762a360 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 --2- 172.21.15.37:0/3542556149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e7613b30 0x5622e762eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 >> 172.21.15.37:0/3542556149 conn(0x5622e7623ee0 msgr2=0x5622e7612420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 shutdown_connections 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.768+0000 7fb5c65ce740 1 -- 172.21.15.37:0/3542556149 wait complete. 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c65ce740 20 client.16101 trim_cache size 0 max 16384 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:14:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 msgr2=0x7fb5a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 0x7fb5a80a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40cdbb0 tx=0x7fb5b406dd90 comp rx=0 tx=0).stop 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 msgr2=0x7fb5bc19d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400c970 tx=0x7fb5b400cd70 comp rx=0 tx=0).stop 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 shutdown_connections 2024-04-25T21:14:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5bc19ce90 0x7fb5bc19d340 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb5a80a3e40 0x7fb5a80a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb5bc1061b0 0x7fb5bc198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 --2- 172.21.15.37:0/2585097805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5bc1010e0 0x7fb5bc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 >> 172.21.15.37:0/2585097805 conn(0x7fb5bc0771d0 msgr2=0x7fb5bc06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 shutdown_connections 2024-04-25T21:14:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:25.769+0000 7fb5c38bb640 1 -- 172.21.15.37:0/2585097805 wait complete. 2024-04-25T21:14:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:25 smithi177 ceph-mon[33043]: pgmap v181: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:25 smithi098 ceph-mon[32569]: pgmap v181: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:25 smithi037 ceph-mon[29195]: pgmap v181: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:27 smithi177 ceph-mon[33043]: pgmap v182: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:28.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:27 smithi098 ceph-mon[32569]: pgmap v182: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:28.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:27 smithi037 ceph-mon[29195]: pgmap v182: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:29 smithi177 ceph-mon[33043]: pgmap v183: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:29 smithi098 ceph-mon[32569]: pgmap v183: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:29 smithi037 ceph-mon[29195]: pgmap v183: 97 pgs: 97 active+clean; 2.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:14:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:31 smithi177 ceph-mon[33043]: pgmap v184: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:31 smithi098 ceph-mon[32569]: pgmap v184: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:31 smithi037 ceph-mon[29195]: pgmap v184: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:33 smithi177 ceph-mon[33043]: pgmap v185: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:34.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:33 smithi098 ceph-mon[32569]: pgmap v185: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:34.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:33 smithi037 ceph-mon[29195]: pgmap v185: 97 pgs: 97 active+clean; 2.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:35 smithi177 ceph-mon[33043]: pgmap v186: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:36.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:35 smithi098 ceph-mon[32569]: pgmap v186: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:35 smithi037 ceph-mon[29195]: pgmap v186: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:37 smithi177 ceph-mon[33043]: pgmap v187: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:37 smithi098 ceph-mon[32569]: pgmap v187: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:37 smithi037 ceph-mon[29195]: pgmap v187: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- 172.21.15.37:0/393056175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc1016a0 msgr2=0x7f90cc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 --2- 172.21.15.37:0/393056175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc1016a0 0x7f90cc10e3f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009900 tx=0x7f90bc02a410 comp rx=0 tx=0).stop 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- 172.21.15.37:0/393056175 shutdown_connections 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 --2- 172.21.15.37:0/393056175 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc10e930 0x7f90cc110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 --2- 172.21.15.37:0/393056175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc1016a0 0x7f90cc10e3f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 --2- 172.21.15.37:0/393056175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc100d80 0x7f90cc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- 172.21.15.37:0/393056175 >> 172.21.15.37:0/393056175 conn(0x7f90cc0771d0 msgr2=0x7f90cc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- 172.21.15.37:0/393056175 shutdown_connections 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- 172.21.15.37:0/393056175 wait complete. 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 Processor -- start 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.870+0000 7f90d3308640 1 -- start start 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc100d80 0x7f90cc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc1016a0 0x7f90cc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc114440 con 0x7f90cc10e930 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1142c0 con 0x7f90cc1016a0 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d3308640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1145c0 con 0x7f90cc100d80 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56906/0 (socket says 172.21.15.37:56906) 2024-04-25T21:14:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 -- 172.21.15.37:0/938471106 learned_addr learned my addr 172.21.15.37:0/938471106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d08a3640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc1016a0 0x7f90cc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d10a4640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc100d80 0x7f90cc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 -- 172.21.15.37:0/938471106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc100d80 msgr2=0x7f90cc1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc100d80 0x7f90cc1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 -- 172.21.15.37:0/938471106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc1016a0 msgr2=0x7f90cc1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc1016a0 0x7f90cc1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 -- 172.21.15.37:0/938471106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc19afd0 con 0x7f90cc10e930 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90d18a5640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f90c400d6a0 tx=0x7f90c400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90c27fc640 1 -- 172.21.15.37:0/938471106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4017070 con 0x7f90cc10e930 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90c27fc640 1 -- 172.21.15.37:0/938471106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c4009820 con 0x7f90cc10e930 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.871+0000 7f90c27fc640 1 -- 172.21.15.37:0/938471106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004c40 con 0x7f90cc10e930 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.872+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc19b2c0 con 0x7f90cc10e930 2024-04-25T21:14:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.872+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc1a74e0 con 0x7f90cc10e930 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.874+0000 7f90c27fc640 1 -- 172.21.15.37:0/938471106 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90c40099d0 con 0x7f90cc10e930 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 Processor -- start 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 -- start start 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e92311a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e92276970 con 0x556e92311f40 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e922134a0 con 0x556e92311620 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d6025740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e92272930 con 0x556e9231c800 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90c27fc640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 0x7f90b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56910/0 (socket says 172.21.15.37:56910) 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d18a5640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 0x7f90b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 -- 172.21.15.37:0/3929888218 learned_addr learned my addr 172.21.15.37:0/3929888218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d08a3640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e92311a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d08a3640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90c27fc640 1 -- 172.21.15.37:0/938471106 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f90c40cffe0 con 0x7f90cc10e930 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 -- 172.21.15.37:0/3929888218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 msgr2=0x556e9231ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 -- 172.21.15.37:0/3929888218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 msgr2=0x556e92311a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.875+0000 7f90d10a4640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e92311a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.876+0000 7f90d10a4640 1 -- 172.21.15.37:0/3929888218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e91f89210 con 0x556e92311f40 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.876+0000 7f90d08a3640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.876+0000 7f90d18a5640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 0x7f90b80a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90c409bf40 tx=0x7f90c406d9f0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.876+0000 7f90d10a4640 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009f60 tx=0x7f90c8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b721098ed77c717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90b77fe640 1 -- 172.21.15.37:0/3929888218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8016070 con 0x556e92311f40 2024-04-25T21:14:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90b77fe640 1 -- 172.21.15.37:0/3929888218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c801bb10 con 0x556e92311f40 2024-04-25T21:14:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90b77fe640 1 -- 172.21.15.37:0/3929888218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8012c30 con 0x556e92311f40 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 -- 172.21.15.37:0/3929888218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 msgr2=0x556e9231c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009f60 tx=0x7f90c8009f90 comp rx=0 tx=0).stop 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 -- 172.21.15.37:0/3929888218 shutdown_connections 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e9231c2c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 --2- 172.21.15.37:0/3929888218 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e92311a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 -- 172.21.15.37:0/3929888218 >> 172.21.15.37:0/3929888218 conn(0x556e9230db30 msgr2=0x556e9230df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 -- 172.21.15.37:0/3929888218 shutdown_connections 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.880+0000 7f90d6025740 1 -- 172.21.15.37:0/3929888218 wait complete. 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 -- start start 2024-04-25T21:14:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 Processor -- start 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e9231c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e92319920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e9231f430 con 0x556e92311f40 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e922110b0 con 0x556e92311620 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e9231f5b0 con 0x556e9231c800 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.881+0000 7f90d6025740 10 client.0 _ll_register_callbacks cb 0x556e921fed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56914/0 (socket says 172.21.15.37:56914) 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 -- 172.21.15.37:0/2330215419 learned_addr learned my addr 172.21.15.37:0/2330215419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e9231c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e92319920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 -- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 msgr2=0x556e92319920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e92319920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 -- 172.21.15.37:0/2330215419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 msgr2=0x556e9231c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e9231c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 -- 172.21.15.37:0/2330215419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e92055620 con 0x556e92311f40 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e9231c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f90c8012680 tx=0x7f90c80126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185f638bdb83c51f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c802b060 con 0x556e92311f40 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f90c8013d00 con 0x556e92311f40 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c801bb10 con 0x556e92311f40 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.882+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90c8016040 con 0x556e92311f40 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 10 client.16113 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e9232b300 con 0x556e92311f40 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c801bb10 con 0x556e92311f40 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90c800f040 con 0x556e92311f40 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 10 client.16113 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 20 client.16113 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e9232c700 0x556e9232eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 4 client.16113 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e92284390 con 0x556e9232c700 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.883+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e9232c700 0x556e9232eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.884+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e9232c700 0x556e9232eaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c93f7a47ab74b1 server_cookie=ec996cfb6721f807 in_seq=0 out_seq=0 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.884+0000 7f90b4ff9640 10 client.16113 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2330215419 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e92284390 con 0x556e9232c700 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90b4ff9640 10 client.16113 handle_command_reply: tid=0 2024-04-25T21:14:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 client.16113 shutdown 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 msgr2=0x556e923193e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f90c8012680 tx=0x7f90c80126b0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 shutdown_connections 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e9232c700 0x556e9232eaf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e92319920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90b4ff9640 0 client.16113 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e923193e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 --2- 172.21.15.37:0/2330215419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e9231c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 >> 172.21.15.37:0/2330215419 conn(0x556e9230db30 msgr2=0x556e923138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 shutdown_connections 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 1 -- 172.21.15.37:0/2330215419 wait complete. 2024-04-25T21:14:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.885+0000 7f90d6025740 20 client.16113 trim_cache size 0 max 16384 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 Processor -- start 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 -- start start 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e923282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1afd40 con 0x556e92311f40 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1b1880 con 0x556e92311620 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.900+0000 7f90d6025740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1b0410 con 0x556e9231c800 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56926/0 (socket says 172.21.15.37:56926) 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 -- 172.21.15.37:0/623491605 learned_addr learned my addr 172.21.15.37:0/623491605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d08a3640 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 -- 172.21.15.37:0/623491605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 msgr2=0x556e9231bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 -- 172.21.15.37:0/623491605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 msgr2=0x556e923282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e923282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 -- 172.21.15.37:0/623491605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9232ef50 con 0x556e92311f40 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d08a3640 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90d10a4640 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f90c8010c90 tx=0x7f90c8010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be890cd2b5307ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90b6ffd640 1 -- 172.21.15.37:0/623491605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c802b040 con 0x556e92311f40 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90b6ffd640 1 -- 172.21.15.37:0/623491605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8009de0 con 0x556e92311f40 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.901+0000 7f90b6ffd640 1 -- 172.21.15.37:0/623491605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c801b6c0 con 0x556e92311f40 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- 172.21.15.37:0/623491605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 msgr2=0x556e92314850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f90c8010c90 tx=0x7f90c8010d40 comp rx=0 tx=0).stop 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- 172.21.15.37:0/623491605 shutdown_connections 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e9231c800 0x556e9231bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e92311f40 0x556e92314850 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 --2- 172.21.15.37:0/623491605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311620 0x556e923282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- 172.21.15.37:0/623491605 >> 172.21.15.37:0/623491605 conn(0x556e9230db30 msgr2=0x556e92314140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- 172.21.15.37:0/623491605 shutdown_connections 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- 172.21.15.37:0/623491605 wait complete. 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 -- start start 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.902+0000 7f90d6025740 1 Processor -- start 2024-04-25T21:14:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e92311620 0x556e92324800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311f40 0x556e92324d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1b11b0 con 0x556e9231c800 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1b0ae0 con 0x556e92311f40 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1add20 con 0x556e92311620 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d6025740 10 client.0 _ll_register_callbacks cb 0x7f90c8009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56932/0 (socket says 172.21.15.37:56932) 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 -- 172.21.15.37:0/2038646321 learned_addr learned my addr 172.21.15.37:0/2038646321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311f40 0x556e92324d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e92311620 0x556e92324800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 -- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e92311620 msgr2=0x556e92324800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e92311620 0x556e92324800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 -- 172.21.15.37:0/2038646321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311f40 msgr2=0x556e92324d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311f40 0x556e92324d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 -- 172.21.15.37:0/2038646321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e92315e80 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90d08a3640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90bc032fd0 tx=0x7f90bc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8318a75382c26aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.903+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc04f060 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f90bc004330 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c8009de0 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f90bc035040 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 10 client.16119 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e92392490 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8009de0 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f90bc031b50 con 0x556e9231c800 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 10 client.16119 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 20 client.16119 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e92392bc0 0x556e92394fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 4 client.16119 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e92319cb0 con 0x556e92392bc0 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.904+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e92392bc0 0x556e92394fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.905+0000 7f90b4ff9640 10 client.16119 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.905+0000 7f90d10a4640 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e92392bc0 0x556e92394fe0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac34cd88cca608e server_cookie=61c7a6716de39c34 in_seq=0 out_seq=0 2024-04-25T21:14:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.919+0000 7f90b4ff9640 1 -- 172.21.15.37:0/2038646321 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e92319cb0 con 0x556e92392bc0 2024-04-25T21:14:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.919+0000 7f90b4ff9640 10 client.16119 handle_command_reply: tid=0 2024-04-25T21:14:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.919+0000 7f90d6025740 1 client.16119 shutdown 2024-04-25T21:14:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 msgr2=0x556e92315770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90bc032fd0 tx=0x7f90bc02a410 comp rx=0 tx=0).stop 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 shutdown_connections 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e92392bc0 0x556e92394fe0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e9231c800 0x556e92315770 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e92311f40 0x556e92324d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 --2- 172.21.15.37:0/2038646321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e92311620 0x556e92324800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90b4ff9640 0 client.16119 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 >> 172.21.15.37:0/2038646321 conn(0x556e9230db30 msgr2=0x556e923149b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 shutdown_connections 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 1 -- 172.21.15.37:0/2038646321 wait complete. 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d6025740 20 client.16119 trim_cache size 0 max 16384 2024-04-25T21:14:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:14:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:14:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:14:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:14:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 msgr2=0x7f90b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 0x7f90b80a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f90c409bf40 tx=0x7f90c406d9f0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 msgr2=0x7f90cc19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.920+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f90c400d6a0 tx=0x7f90c400db60 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 shutdown_connections 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90cc10e930 0x7f90cc19a800 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90cc1016a0 0x7f90cc1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f90b80a3e40 0x7f90b80a6300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 --2- 172.21.15.37:0/938471106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90cc100d80 0x7f90cc1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 >> 172.21.15.37:0/938471106 conn(0x7f90cc0771d0 msgr2=0x7f90cc10fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 shutdown_connections 2024-04-25T21:14:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:38.921+0000 7f90d3308640 1 -- 172.21.15.37:0/938471106 wait complete. 2024-04-25T21:14:39.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:38 smithi098 ceph-mon[32569]: pgmap v188: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:39.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:38 smithi037 ceph-mon[29195]: pgmap v188: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:39.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:38 smithi177 ceph-mon[33043]: pgmap v188: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:14:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:14:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:14:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:14:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:39 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:14:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:14:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:14:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:14:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:39 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:14:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:14:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:14:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:14:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:39 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:14:41.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:40 smithi098 ceph-mon[32569]: pgmap v189: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-25T21:14:41.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:40 smithi037 ceph-mon[29195]: pgmap v189: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-25T21:14:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:40 smithi177 ceph-mon[33043]: pgmap v189: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-25T21:14:43.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:43 smithi177 ceph-mon[33043]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:43.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:43 smithi098 ceph-mon[32569]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:43.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:43 smithi037 ceph-mon[29195]: pgmap v190: 97 pgs: 97 active+clean; 2.7 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:45.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:45 smithi177 ceph-mon[33043]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T21:14:45.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:45 smithi098 ceph-mon[32569]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T21:14:45.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:45 smithi037 ceph-mon[29195]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-25T21:14:47.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:47 smithi177 ceph-mon[33043]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:47.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:47 smithi098 ceph-mon[32569]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:47.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:47 smithi037 ceph-mon[29195]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:49.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:49 smithi177 ceph-mon[33043]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:49.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:49 smithi098 ceph-mon[32569]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:49.669 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:127/600: open fd = 3 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:128/600: open fd = 3 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:129/600: open fd = 3 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.670 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:130/600: open fd = 3 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:131/600: open fd = 3 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:132/600: open fd = 3 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.671 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:133/600: open fd = 3 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:134/600: open fd = 3 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:135/600: open fd = 3 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:136/600: open fd = 3 2024-04-25T21:14:49.672 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:137/600: open fd = 3 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:138/600: open fd = 3 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:139/600: open fd = 3 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.673 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:140/600: open fd = 3 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:141/600: open fd = 3 2024-04-25T21:14:49.674 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:142/600: open fd = 3 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:143/600: open fd = 3 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:144/600: open fd = 3 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.675 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:145/600: open fd = 3 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:146/600: open fd = 3 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:147/600: open fd = 3 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.676 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:148/600: open fd = 3 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:149/600: open fd = 3 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:150/600: open fd = 3 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:151/600: open fd = 3 2024-04-25T21:14:49.677 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:152/600: open fd = 3 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:153/600: open fd = 3 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:154/600: open fd = 3 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.678 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:155/600: open fd = 3 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:156/600: open fd = 3 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:157/600: open fd = 3 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.679 INFO:tasks.workunit.client.0.smithi037.stdout:158/600: open fd = 3 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:159/600: open fd = 3 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:160/600: open fd = 3 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:161/600: open fd = 3 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.680 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:162/600: open fd = 3 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:163/600: open fd = 3 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:164/600: open fd = 3 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.681 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:165/600: open fd = 3 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:166/600: open fd = 3 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:167/600: open fd = 3 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:168/600: open fd = 3 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.682 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:169/600: open fd = 3 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:170/600: open fd = 3 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:171/600: open fd = 3 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:172/600: open fd = 3 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.683 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:173/600: open fd = 3 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:174/600: open fd = 3 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:175/600: open fd = 3 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.684 INFO:tasks.workunit.client.0.smithi037.stdout:176/600: open fd = 3 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:177/600: open fd = 3 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:178/600: open fd = 3 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:179/600: open fd = 3 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.685 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:180/600: open fd = 3 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:181/600: open fd = 3 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:182/600: open fd = 3 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:183/600: open fd = 3 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.686 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:184/600: open fd = 3 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:185/600: open fd = 3 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:186/600: open fd = 3 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:187/600: open fd = 3 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.687 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.688 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:14:49.688 INFO:tasks.workunit.client.0.smithi037.stdout:188/600: open fd = 3 2024-04-25T21:14:49.688 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.688 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:14:49.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:49 smithi037 ceph-mon[29195]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:14:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:51 smithi177 ceph-mon[33043]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:51.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:51 smithi098 ceph-mon[32569]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:51.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:51 smithi037 ceph-mon[29195]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- 172.21.15.37:0/3034427139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6704103ac0 msgr2=0x7f670410bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 --2- 172.21.15.37:0/3034427139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6704103ac0 0x7f670410bfa0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009900 tx=0x7f66f802a480 comp rx=0 tx=0).stop 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- 172.21.15.37:0/3034427139 shutdown_connections 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 --2- 172.21.15.37:0/3034427139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f670410c570 0x7f670410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 --2- 172.21.15.37:0/3034427139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6704103ac0 0x7f670410bfa0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 --2- 172.21.15.37:0/3034427139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f67041031a0 0x7f6704103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- 172.21.15.37:0/3034427139 >> 172.21.15.37:0/3034427139 conn(0x7f6704078060 msgr2=0x7f6704078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- 172.21.15.37:0/3034427139 shutdown_connections 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- 172.21.15.37:0/3034427139 wait complete. 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 Processor -- start 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.020+0000 7f670accf640 1 -- start start 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67041031a0 0x7f67041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6704103ac0 0x7f6704198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67041121b0 con 0x7f670410c570 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6704112030 con 0x7f6704103ac0 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670accf640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6704112330 con 0x7f67041031a0 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48158/0 (socket says 172.21.15.37:48158) 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 -- 172.21.15.37:0/3666544966 learned_addr learned my addr 172.21.15.37:0/3666544966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f6703fff640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6704103ac0 0x7f6704198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f6708a6b640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67041031a0 0x7f67041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 -- 172.21.15.37:0/3666544966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67041031a0 msgr2=0x7f67041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67041031a0 0x7f67041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 -- 172.21.15.37:0/3666544966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6704103ac0 msgr2=0x7f6704198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6704103ac0 0x7f6704198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 -- 172.21.15.37:0/3666544966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670419d670 con 0x7f670410c570 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.021+0000 7f670926c640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e960 tx=0x7f66fc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.022+0000 7f6701ffb640 1 -- 172.21.15.37:0/3666544966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc004d20 con 0x7f670410c570 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.022+0000 7f6701ffb640 1 -- 172.21.15.37:0/3666544966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66fc005020 con 0x7f670410c570 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.022+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670419d960 con 0x7f670410c570 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.022+0000 7f6701ffb640 1 -- 172.21.15.37:0/3666544966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0116d0 con 0x7f670410c570 2024-04-25T21:14:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.022+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a51f0 con 0x7f670410c570 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.024+0000 7f6701ffb640 1 -- 172.21.15.37:0/3666544966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66fc0051d0 con 0x7f670410c570 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 Processor -- start 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 -- start start 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6701ffb640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 0x7f66ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 0x5595804a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55958040d970 con 0x5595804a8f40 2024-04-25T21:14:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5595803aa4a0 con 0x5595804b3800 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670d9eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559580409930 con 0x5595804a8620 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6701ffb640 1 -- 172.21.15.37:0/3666544966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f66fc0d0760 con 0x7f670410c570 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6703fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6703fff640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 0x7f66ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6703fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48174/0 (socket says 172.21.15.37:48174) 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6703fff640 1 -- 172.21.15.37:0/2159884696 learned_addr learned my addr 172.21.15.37:0/2159884696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f6708a6b640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 0x5595804a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.025+0000 7f670926c640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 -- 172.21.15.37:0/2159884696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 msgr2=0x5595804a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 0x5595804a8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 -- 172.21.15.37:0/2159884696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 msgr2=0x5595804b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 -- 172.21.15.37:0/2159884696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559580120210 con 0x5595804a8f40 2024-04-25T21:14:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6708a6b640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 0x5595804a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f670926c640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 0x7f66ec0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66f802a9a0 tx=0x7f66f8005bc0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f6703fff640 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f66f8033000 tx=0x7f66f8046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508e1311fcd5126e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f66eaffd640 1 -- 172.21.15.37:0/2159884696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f804d070 con 0x5595804a8f40 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f66eaffd640 1 -- 172.21.15.37:0/2159884696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f8003ca0 con 0x5595804a8f40 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.026+0000 7f66eaffd640 1 -- 172.21.15.37:0/2159884696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8035070 con 0x5595804a8f40 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- 172.21.15.37:0/2159884696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 msgr2=0x5595804b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f66f8033000 tx=0x7f66f8046000 comp rx=0 tx=0).stop 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- 172.21.15.37:0/2159884696 shutdown_connections 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8f40 0x5595804b32c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2159884696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8620 0x5595804a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- 172.21.15.37:0/2159884696 >> 172.21.15.37:0/2159884696 conn(0x5595804a4b30 msgr2=0x5595804a4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- 172.21.15.37:0/2159884696 shutdown_connections 2024-04-25T21:14:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- 172.21.15.37:0/2159884696 wait complete. 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 -- start start 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.027+0000 7f670d9eb740 1 Processor -- start 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8620 0x5595804b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5595804b63d0 con 0x5595804b3800 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5595803a80b0 con 0x5595804a8620 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5595804b6550 con 0x5595804a8f40 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670d9eb740 10 client.0 _ll_register_callbacks cb 0x559580395d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48186/0 (socket says 172.21.15.37:48186) 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 -- 172.21.15.37:0/2820811801 learned_addr learned my addr 172.21.15.37:0/2820811801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f6708a6b640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8620 0x5595804b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f6703fff640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 -- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 msgr2=0x5595804b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 -- 172.21.15.37:0/2820811801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8620 msgr2=0x5595804b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8620 0x5595804b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 -- 172.21.15.37:0/2820811801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595801ec620 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f670926c640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c640 tx=0x7f66fc09c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca2be074e53c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f66e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.028+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66fc01f040 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f66fc098e50 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f8003ca0 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66fc0054f0 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 10 client.16131 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5595804c2360 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f8003ca0 con 0x5595804b3800 2024-04-25T21:14:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66fc00f040 con 0x5595804b3800 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 20 client.16131 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5595804c37d0 0x5595804c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 4 client.16131 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.029+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55958041b390 con 0x5595804c37d0 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.030+0000 7f6708a6b640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5595804c37d0 0x5595804c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.030+0000 7f6708a6b640 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5595804c37d0 0x5595804c5be0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506fb22ce877e11d server_cookie=dc843e31ea110323 in_seq=0 out_seq=0 2024-04-25T21:14:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.030+0000 7f66e3fff640 10 client.16131 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f66e3fff640 1 -- 172.21.15.37:0/2820811801 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55958041b390 con 0x5595804c37d0 2024-04-25T21:14:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f66e3fff640 10 client.16131 handle_command_reply: tid=0 2024-04-25T21:14:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 client.16131 shutdown 2024-04-25T21:14:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 msgr2=0x5595804b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00c640 tx=0x7f66fc09c850 comp rx=0 tx=0).stop 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 shutdown_connections 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5595804c37d0 0x5595804c5be0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804b3800 0x5595804b09f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 --2- 172.21.15.37:0/2820811801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8620 0x5595804b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f66e3fff640 0 client.16131 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 >> 172.21.15.37:0/2820811801 conn(0x5595804a4b30 msgr2=0x5595804aa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 shutdown_connections 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 1 -- 172.21.15.37:0/2820811801 wait complete. 2024-04-25T21:14:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.031+0000 7f670d9eb740 20 client.16131 trim_cache size 0 max 16384 2024-04-25T21:14:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 Processor -- start 2024-04-25T21:14:52.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 -- start start 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8f40 0x5595804cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804b3800 0x5595804cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ada60 con 0x5595804a8620 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67041af5a0 con 0x5595804a8f40 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.046+0000 7f670d9eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ae130 con 0x5595804b3800 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48196/0 (socket says 172.21.15.37:48196) 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 -- 172.21.15.37:0/408244651 learned_addr learned my addr 172.21.15.37:0/408244651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f670926c640 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804b3800 0x5595804cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6703fff640 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8f40 0x5595804cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 -- 172.21.15.37:0/408244651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804b3800 msgr2=0x5595804cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804b3800 0x5595804cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 -- 172.21.15.37:0/408244651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8f40 msgr2=0x5595804cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8f40 0x5595804cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 -- 172.21.15.37:0/408244651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595804a8d70 con 0x5595804a8620 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f6708a6b640 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5595804c64e0 tx=0x7f66f400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faac3d9399505c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f66ea7fc640 1 -- 172.21.15.37:0/408244651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4004220 con 0x5595804a8620 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f66ea7fc640 1 -- 172.21.15.37:0/408244651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f40043c0 con 0x5595804a8620 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.047+0000 7f66ea7fc640 1 -- 172.21.15.37:0/408244651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4003bc0 con 0x5595804a8620 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- 172.21.15.37:0/408244651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 msgr2=0x5595804bf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x5595804c64e0 tx=0x7f66f400b410 comp rx=0 tx=0).stop 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- 172.21.15.37:0/408244651 shutdown_connections 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804b3800 0x5595804cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804a8f40 0x5595804cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 --2- 172.21.15.37:0/408244651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804bf290 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- 172.21.15.37:0/408244651 >> 172.21.15.37:0/408244651 conn(0x5595804a4b30 msgr2=0x5595804a3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- 172.21.15.37:0/408244651 shutdown_connections 2024-04-25T21:14:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- 172.21.15.37:0/408244651 wait complete. 2024-04-25T21:14:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 -- start start 2024-04-25T21:14:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.048+0000 7f670d9eb740 1 Processor -- start 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67041aeed0 con 0x5595804a8620 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67041ae800 con 0x5595804b3800 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f67041aba40 con 0x5595804a8f40 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f670d9eb740 10 client.0 _ll_register_callbacks cb 0x7f66fc09c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48202/0 (socket says 172.21.15.37:48202) 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 -- 172.21.15.37:0/1164029028 learned_addr learned my addr 172.21.15.37:0/1164029028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 -- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 msgr2=0x5595804b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 -- 172.21.15.37:0/1164029028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 msgr2=0x5595804b9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 -- 172.21.15.37:0/1164029028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559580529010 con 0x5595804a8620 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f6708a6b640 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f66f40057c0 tx=0x7f66f400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52e8ab850739bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.049+0000 7f66e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f402d070 con 0x5595804a8620 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8469+0+0 (secure 0 0 0) 0x7f66f4005050 con 0x5595804a8620 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f40043c0 con 0x5595804a8620 2024-04-25T21:14:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66f4039050 con 0x5595804a8620 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 10 client.16137 fetch_fsmap learned FSMap version 27 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55958052a300 con 0x5595804a8620 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f40043c0 con 0x5595804a8620 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f66f40047c0 con 0x5595804a8620 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 20 client.16137 populate_metadata read hostname 'smithi037' 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55958052a960 0x55958052ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 4 client.16137 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559580415200 con 0x55958052a960 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.050+0000 7f6703fff640 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55958052a960 0x55958052ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.051+0000 7f66e3fff640 10 client.16137 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.051+0000 7f6703fff640 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55958052a960 0x55958052ce00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d69c2291eedbc52 server_cookie=85d64e1dd9ee01da in_seq=0 out_seq=0 2024-04-25T21:14:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.066+0000 7f66e3fff640 1 -- 172.21.15.37:0/1164029028 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559580415200 con 0x55958052a960 2024-04-25T21:14:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.066+0000 7f66e3fff640 10 client.16137 handle_command_reply: tid=0 2024-04-25T21:14:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.066+0000 7f670d9eb740 1 client.16137 shutdown 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.066+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 msgr2=0x5595804b9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.066+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f66f40057c0 tx=0x7f66f400bcb0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 shutdown_connections 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55958052a960 0x55958052ce00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5595804b3800 0x5595804b9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f66e3fff640 0 client.16137 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5595804a8f40 0x5595804b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 --2- 172.21.15.37:0/1164029028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5595804a8620 0x5595804b9470 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 >> 172.21.15.37:0/1164029028 conn(0x5595804a4b30 msgr2=0x5595804be280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 shutdown_connections 2024-04-25T21:14:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 1 -- 172.21.15.37:0/1164029028 wait complete. 2024-04-25T21:14:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670d9eb740 20 client.16137 trim_cache size 0 max 16384 2024-04-25T21:14:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:14:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:14:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:14:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 msgr2=0x7f66ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 0x7f66ec0a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66f802a9a0 tx=0x7f66f8005bc0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 msgr2=0x7f670419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.067+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f66fc00e960 tx=0x7f66fc00ee20 comp rx=0 tx=0).stop 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 shutdown_connections 2024-04-25T21:14:52.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670410c570 0x7f670419cf40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6704103ac0 0x7f6704198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f66ec0a3e40 0x7f66ec0a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 --2- 172.21.15.37:0/3666544966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f67041031a0 0x7f67041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 >> 172.21.15.37:0/3666544966 conn(0x7f6704078060 msgr2=0x7f67040fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 shutdown_connections 2024-04-25T21:14:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:14:52.068+0000 7f670accf640 1 -- 172.21.15.37:0/3666544966 wait complete. 2024-04-25T21:14:53.537 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:53 smithi037 ceph-mon[29195]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T21:14:53.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:53 smithi177 ceph-mon[33043]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T21:14:53.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:53 smithi098 ceph-mon[32569]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 13 op/s 2024-04-25T21:14:55.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:55 smithi177 ceph-mon[33043]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:55.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:55 smithi098 ceph-mon[32569]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:55.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:55 smithi037 ceph-mon[29195]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:14:57.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:57 smithi098 ceph-mon[32569]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:57.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:57 smithi037 ceph-mon[29195]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:57 smithi177 ceph-mon[33043]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:59.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:14:59 smithi098 ceph-mon[32569]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:59.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:14:59 smithi037 ceph-mon[29195]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:14:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:14:59 smithi177 ceph-mon[33043]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:15:01.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:01 smithi098 ceph-mon[32569]: pgmap v199: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:15:01.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:01 smithi037 ceph-mon[29195]: pgmap v199: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:15:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:01 smithi177 ceph-mon[33043]: pgmap v199: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:15:02.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:02 smithi098 ceph-mon[32569]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T21:15:02.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:02 smithi037 ceph-mon[29195]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T21:15:02.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:02 smithi177 ceph-mon[33043]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T21:15:03.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:03 smithi098 ceph-mon[32569]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:15:03.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:03 smithi037 ceph-mon[29195]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:15:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:03 smithi177 ceph-mon[33043]: pgmap v201: 97 pgs: 97 active+clean; 3.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:15:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 -- 172.21.15.37:0/3277595915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a41016a0 msgr2=0x7f47a410e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 --2- 172.21.15.37:0/3277595915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a41016a0 0x7f47a410e3f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c02a410 comp rx=0 tx=0).stop 2024-04-25T21:15:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 -- 172.21.15.37:0/3277595915 shutdown_connections 2024-04-25T21:15:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 --2- 172.21.15.37:0/3277595915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a410e930 0x7f47a4110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 --2- 172.21.15.37:0/3277595915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a41016a0 0x7f47a410e3f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 --2- 172.21.15.37:0/3277595915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a4100d80 0x7f47a4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 -- 172.21.15.37:0/3277595915 >> 172.21.15.37:0/3277595915 conn(0x7f47a4077fd0 msgr2=0x7f47a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 -- 172.21.15.37:0/3277595915 shutdown_connections 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 -- 172.21.15.37:0/3277595915 wait complete. 2024-04-25T21:15:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.169+0000 7f47a9787640 1 Processor -- start 2024-04-25T21:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 -- start start 2024-04-25T21:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a41016a0 0x7f47a41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a410e930 0x7f47a419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41143c0 con 0x7f47a4100d80 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4114240 con 0x7f47a41016a0 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a9787640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4114540 con 0x7f47a410e930 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46374/0 (socket says 172.21.15.37:46374) 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a2ffd640 1 -- 172.21.15.37:0/3835853863 learned_addr learned my addr 172.21.15.37:0/3835853863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f47a37fe640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a410e930 0x7f47a419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.170+0000 7f479a7fc640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a41016a0 0x7f47a41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 -- 172.21.15.37:0/3835853863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a410e930 msgr2=0x7f47a419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a410e930 0x7f47a419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 -- 172.21.15.37:0/3835853863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a41016a0 msgr2=0x7f47a41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a41016a0 0x7f47a41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 -- 172.21.15.37:0/3835853863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a419afd0 con 0x7f47a4100d80 2024-04-25T21:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f479400d950 tx=0x7f479400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a0ff9640 1 -- 172.21.15.37:0/3835853863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794017070 con 0x7f47a4100d80 2024-04-25T21:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a419b2c0 con 0x7f47a4100d80 2024-04-25T21:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a0ff9640 1 -- 172.21.15.37:0/3835853863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4794009c30 con 0x7f47a4100d80 2024-04-25T21:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a0ff9640 1 -- 172.21.15.37:0/3835853863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400f620 con 0x7f47a4100d80 2024-04-25T21:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.171+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41a74e0 con 0x7f47a4100d80 2024-04-25T21:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.173+0000 7f47a0ff9640 1 -- 172.21.15.37:0/3835853863 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4794010460 con 0x7f47a4100d80 2024-04-25T21:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 Processor -- start 2024-04-25T21:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 -- start start 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47a0ff9640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211620 0x55635c211a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55635c176970 con 0x55635c21c800 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55635c1134a0 con 0x55635c211620 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47ac57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55635c172930 con 0x55635c211f40 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47a37fe640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47a0ff9640 1 -- 172.21.15.37:0/3835853863 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f47940d0410 con 0x7f47a4100d80 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f479a7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f479a7fc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:41820/0 (socket says 172.21.15.37:41820) 2024-04-25T21:15:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f479a7fc640 1 -- 172.21.15.37:0/3372021502 learned_addr learned my addr 172.21.15.37:0/3372021502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.174+0000 7f47a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46384/0 (socket says 172.21.15.37:46384) 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211620 0x55635c211a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 -- 172.21.15.37:0/3372021502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211620 msgr2=0x55635c211a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211620 0x55635c211a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 -- 172.21.15.37:0/3372021502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 msgr2=0x55635c21ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 -- 172.21.15.37:0/3372021502 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55635be89210 con 0x55635c211f40 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47a37fe640 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47a37fe640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 0x7f47900a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55635c20ef50 tx=0x7f479c004ab0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f479a7fc640 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e525693c61addc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47997fa640 1 -- 172.21.15.37:0/3372021502 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c040070 con 0x55635c211f40 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47997fa640 1 -- 172.21.15.37:0/3372021502 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c004030 con 0x55635c211f40 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.175+0000 7f47997fa640 1 -- 172.21.15.37:0/3372021502 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c035040 con 0x55635c211f40 2024-04-25T21:15:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 -- 172.21.15.37:0/3372021502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 msgr2=0x55635c21c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c032bd0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 -- 172.21.15.37:0/3372021502 shutdown_connections 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c21c800 0x55635c21ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c21c2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 --2- 172.21.15.37:0/3372021502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211620 0x55635c211a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 -- 172.21.15.37:0/3372021502 >> 172.21.15.37:0/3372021502 conn(0x55635c20db30 msgr2=0x55635c20df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 -- 172.21.15.37:0/3372021502 shutdown_connections 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.176+0000 7f47ac57b740 1 -- 172.21.15.37:0/3372021502 wait complete. 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 -- start start 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 Processor -- start 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c2194b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c219ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55635c21f3d0 con 0x55635c211620 2024-04-25T21:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55635c1110b0 con 0x55635c21c800 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55635c21f550 con 0x55635c211f40 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47ac57b740 10 client.0 _ll_register_callbacks cb 0x55635c0fed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46400/0 (socket says 172.21.15.37:46400) 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 -- 172.21.15.37:0/286066408 learned_addr learned my addr 172.21.15.37:0/286066408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f479a7fc640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c2194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a37fe640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c219ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 -- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 msgr2=0x55635c2194b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c2194b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 -- 172.21.15.37:0/286066408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 msgr2=0x55635c219ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c219ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a2ffd640 1 -- 172.21.15.37:0/286066408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635bf55620 con 0x55635c211620 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.177+0000 7f47a37fe640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c219ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47940987f0 tx=0x7f47940b5d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32c3a5eccacc19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794020070 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f47940cdd60 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478c004030 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f47940cdf60 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 10 client.16149 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55635c22b360 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c004030 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f47940a0160 con 0x55635c211620 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 10 client.16149 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 20 client.16149 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c22c820 0x55635c22ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 4 client.16149 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.178+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55635c184390 con 0x55635c22c820 2024-04-25T21:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.179+0000 7f479a7fc640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c22c820 0x55635c22ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.179+0000 7f479a7fc640 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c22c820 0x55635c22ec10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b491769821bed658 server_cookie=9f191537531df748 in_seq=0 out_seq=0 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.179+0000 7f478a7fc640 10 client.16149 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f478a7fc640 1 -- 172.21.15.37:0/286066408 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55635c184390 con 0x55635c22c820 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f478a7fc640 10 client.16149 handle_command_reply: tid=0 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 client.16149 shutdown 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 msgr2=0x55635c21c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f47940987f0 tx=0x7f47940b5d50 comp rx=0 tx=0).stop 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 shutdown_connections 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c22c820 0x55635c22ec10 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c219ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f478a7fc640 0 client.16149 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211f40 0x55635c2194b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 --2- 172.21.15.37:0/286066408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c21c1c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 >> 172.21.15.37:0/286066408 conn(0x55635c20db30 msgr2=0x55635c213860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 shutdown_connections 2024-04-25T21:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 1 -- 172.21.15.37:0/286066408 wait complete. 2024-04-25T21:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.180+0000 7f47ac57b740 20 client.16149 trim_cache size 0 max 16384 2024-04-25T21:15:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.195+0000 7f47ac57b740 1 Processor -- start 2024-04-25T21:15:05.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.195+0000 7f47ac57b740 1 -- start start 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211620 0x55635c228290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c2344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41afda0 con 0x55635c211f40 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b18e0 con 0x55635c21c800 2024-04-25T21:15:05.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47ac57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b0470 con 0x55635c211620 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46410/0 (socket says 172.21.15.37:46410) 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 -- 172.21.15.37:0/2307164706 learned_addr learned my addr 172.21.15.37:0/2307164706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211620 0x55635c228290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47a37fe640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c2344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 -- 172.21.15.37:0/2307164706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211620 msgr2=0x55635c228290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211620 0x55635c228290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 -- 172.21.15.37:0/2307164706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 msgr2=0x55635c2344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c2344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 -- 172.21.15.37:0/2307164706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55635c21ee20 con 0x55635c211f40 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f47a37fe640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c2344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f479a7fc640 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd6ff3e58c68bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.196+0000 7f4798ff9640 1 -- 172.21.15.37:0/2307164706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c040040 con 0x55635c211f40 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f4798ff9640 1 -- 172.21.15.37:0/2307164706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c0314a0 con 0x55635c211f40 2024-04-25T21:15:05.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f4798ff9640 1 -- 172.21.15.37:0/2307164706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478c035040 con 0x55635c211f40 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 -- 172.21.15.37:0/2307164706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 msgr2=0x55635c20cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f478c009900 tx=0x7f478c031400 comp rx=0 tx=0).stop 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 -- 172.21.15.37:0/2307164706 shutdown_connections 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c21c800 0x55635c2344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211f40 0x55635c20cb30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 --2- 172.21.15.37:0/2307164706 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c211620 0x55635c228290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 -- 172.21.15.37:0/2307164706 >> 172.21.15.37:0/2307164706 conn(0x55635c20db30 msgr2=0x55635c20c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 -- 172.21.15.37:0/2307164706 shutdown_connections 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.197+0000 7f47ac57b740 1 -- 172.21.15.37:0/2307164706 wait complete. 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 -- start start 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 Processor -- start 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211f40 0x55635c221e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 0x55635c222370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b1210 con 0x55635c211620 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41b0b40 con 0x55635c211f40 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41adda0 con 0x55635c21c800 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47ac57b740 10 client.0 _ll_register_callbacks cb 0x7f479409bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46422/0 (socket says 172.21.15.37:46422) 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 -- 172.21.15.37:0/1299109775 learned_addr learned my addr 172.21.15.37:0/1299109775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a37fe640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 0x55635c222370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f479a7fc640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211f40 0x55635c221e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 -- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 msgr2=0x55635c222370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 0x55635c222370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 -- 172.21.15.37:0/1299109775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211f40 msgr2=0x55635c221e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211f40 0x55635c221e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a2ffd640 1 -- 172.21.15.37:0/1299109775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635c222b70 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.198+0000 7f47a37fe640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 0x55635c222370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47a2ffd640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47940a0550 tx=0x7f47940b5b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2dc1a59b546617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794014070 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8557+0+0 (secure 0 0 0) 0x7f4794098380 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f478c0314a0 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4794085050 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55635c29b440 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f478c0314a0 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f479407e060 con 0x55635c211620 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 20 client.16155 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c29bb50 0x55635c29df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 4 client.16155 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.199+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55635c17e200 con 0x55635c29bb50 2024-04-25T21:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.200+0000 7f479a7fc640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c29bb50 0x55635c29df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.200+0000 7f479a7fc640 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c29bb50 0x55635c29df40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f957957be4d8de6 server_cookie=c9f813e953e95c02 in_seq=0 out_seq=0 2024-04-25T21:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.200+0000 7f478a7fc640 10 client.16155 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.215+0000 7f478a7fc640 1 -- 172.21.15.37:0/1299109775 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55635c17e200 con 0x55635c29bb50 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.215+0000 7f478a7fc640 10 client.16155 handle_command_reply: tid=0 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 client.16155 shutdown 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 msgr2=0x55635c224aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f47940a0550 tx=0x7f47940b5b70 comp rx=0 tx=0).stop 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 shutdown_connections 2024-04-25T21:15:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55635c29bb50 0x55635c29df40 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55635c21c800 0x55635c222370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55635c211f40 0x55635c221e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f478a7fc640 0 client.16155 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 --2- 172.21.15.37:0/1299109775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55635c211620 0x55635c224aa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 >> 172.21.15.37:0/1299109775 conn(0x55635c20db30 msgr2=0x55635c20ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 shutdown_connections 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 1 -- 172.21.15.37:0/1299109775 wait complete. 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47ac57b740 20 client.16155 trim_cache size 0 max 16384 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:15:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:15:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 msgr2=0x7f47900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.216+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 0x7f47900a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55635c20ef50 tx=0x7f479c004ab0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 msgr2=0x7f47a41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f479400d950 tx=0x7f479400de10 comp rx=0 tx=0).stop 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 shutdown_connections 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47a410e930 0x7f47a419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47a41016a0 0x7f47a41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 --2- 172.21.15.37:0/3835853863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47a4100d80 0x7f47a41a0690 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 >> 172.21.15.37:0/3835853863 conn(0x7f47a4077fd0 msgr2=0x7f47a410ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 shutdown_connections 2024-04-25T21:15:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:05.217+0000 7f47a9787640 1 -- 172.21.15.37:0/3835853863 wait complete. 2024-04-25T21:15:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:05 smithi098 ceph-mon[32569]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-04-25T21:15:05.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:05 smithi098 ceph-mon[32569]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:15:05.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:05 smithi037 ceph-mon[29195]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-04-25T21:15:05.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:05 smithi037 ceph-mon[29195]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:15:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:05 smithi177 ceph-mon[33043]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-04-25T21:15:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:05 smithi177 ceph-mon[33043]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T21:15:07.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:07 smithi098 ceph-mon[32569]: pgmap v204: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:07.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:07 smithi037 ceph-mon[29195]: pgmap v204: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:07 smithi177 ceph-mon[33043]: pgmap v204: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:09.666 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:09 smithi098 ceph-mon[32569]: pgmap v205: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-04-25T21:15:09.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:09 smithi037 ceph-mon[29195]: pgmap v205: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-04-25T21:15:09.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:09 smithi177 ceph-mon[33043]: pgmap v205: 97 pgs: 97 active+clean; 3.7 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 95 KiB/s wr, 35 op/s 2024-04-25T21:15:10.915 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:15:10.916 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-04-25T21:15:10.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:15:11.180 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:11.566 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:11 smithi037 ceph-mon[29195]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:11.667 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:11 smithi098 ceph-mon[32569]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:11.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:11 smithi177 ceph-mon[33043]: pgmap v206: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 118 KiB/s wr, 44 op/s 2024-04-25T21:15:13.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 -- 172.21.15.37:0/3430607244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee01022f0 msgr2=0x7f8ee010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 --2- 172.21.15.37:0/3430607244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee01022f0 0x7f8ee010e7e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800b0a0 tx=0x7f8ed800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 -- 172.21.15.37:0/3430607244 shutdown_connections 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 --2- 172.21.15.37:0/3430607244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee01022f0 0x7f8ee010e7e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 --2- 172.21.15.37:0/3430607244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee0107980 0x7f8ee0107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 --2- 172.21.15.37:0/3430607244 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 -- 172.21.15.37:0/3430607244 >> 172.21.15.37:0/3430607244 conn(0x7f8ee00788b0 msgr2=0x7f8ee0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 -- 172.21.15.37:0/3430607244 shutdown_connections 2024-04-25T21:15:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.100+0000 7f8ee8022700 1 -- 172.21.15.37:0/3430607244 wait complete. 2024-04-25T21:15:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 Processor -- start 2024-04-25T21:15:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 -- start start 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 0x7f8ee0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111af0 con 0x7f8ee0107980 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111970 con 0x7f8ee0101930 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee8022700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111c70 con 0x7f8ee01022f0 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee65bf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee55bd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 0x7f8ee0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee65bf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54556/0 (socket says 172.21.15.37:54556) 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee65bf700 1 -- 172.21.15.37:0/748957397 learned_addr learned my addr 172.21.15.37:0/748957397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.101+0000 7f8ee5dbe700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 -- 172.21.15.37:0/748957397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 msgr2=0x7f8ee0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 0x7f8ee0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 -- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 msgr2=0x7f8ee019da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:13.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 -- 172.21.15.37:0/748957397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee019e140 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee65bf700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee55bd700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 0x7f8ee0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee5dbe700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0199130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009a30 tx=0x7f8ed4009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004830 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee019e430 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee0103410 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.102+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed4005020 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.103+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4007910 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.103+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec4005330 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.105+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ed40049d0 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.106+0000 7f8ed2ffd700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 0x7f8ecc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.106+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f8ed40d4240 con 0x7f8ee0101930 2024-04-25T21:15:13.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.106+0000 7f8ee55bd700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 0x7f8ecc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.106+0000 7f8ee55bd700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 0x7f8ecc0a6270 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0079000 tx=0x7f8edc00a300 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:13.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.110+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ed40a08e0 con 0x7f8ee0101930 2024-04-25T21:15:13.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.365+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ec4005620 con 0x7f8ee0101930 2024-04-25T21:15:13.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.366+0000 7f8ed2ffd700 1 -- 172.21.15.37:0/748957397 <== mon.2 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8ed40a01e0 con 0x7f8ee0101930 2024-04-25T21:15:13.366 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:15:13.366 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 msgr2=0x7f8ecc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 0x7f8ecc0a6270 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0079000 tx=0x7f8edc00a300 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 msgr2=0x7f8ee0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0199130 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009a30 tx=0x7f8ed4009d40 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 shutdown_connections 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f8ecc0a3db0 0x7f8ecc0a6270 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ee0107980 0x7f8ee019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ee0101930 0x7f8ee0199130 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 --2- 172.21.15.37:0/748957397 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ee01022f0 0x7f8ee0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 >> 172.21.15.37:0/748957397 conn(0x7f8ee00788b0 msgr2=0x7f8ee006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 shutdown_connections 2024-04-25T21:15:13.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:13.369+0000 7f8ee8022700 1 -- 172.21.15.37:0/748957397 wait complete. 2024-04-25T21:15:13.370 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:15:13.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:13 smithi037 ceph-mon[29195]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:15:13.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:13 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/748957397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:13.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:13 smithi177 ceph-mon[33043]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:15:13.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:13 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/748957397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:13.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:13 smithi098 ceph-mon[32569]: pgmap v207: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:15:13.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:13 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/748957397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:14.092 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0' 2024-04-25T21:15:14.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0 2024-04-25T21:15:14.350 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:15.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:15 smithi037 ceph-mon[29195]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:15:15.719 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:15 smithi098 ceph-mon[32569]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:15:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:15 smithi177 ceph-mon[33043]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- 172.21.15.37:0/2774652536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806b1b0 msgr2=0x7fd6a806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 --2- 172.21.15.37:0/2774652536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806b1b0 0x7fd6a806b630 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd694009990 tx=0x7fd694009ca0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- 172.21.15.37:0/2774652536 shutdown_connections 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 --2- 172.21.15.37:0/2774652536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806bb70 0x7fd6a810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 --2- 172.21.15.37:0/2774652536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 0x7fd6a8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 --2- 172.21.15.37:0/2774652536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806b1b0 0x7fd6a806b630 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- 172.21.15.37:0/2774652536 >> 172.21.15.37:0/2774652536 conn(0x7fd6a8077990 msgr2=0x7fd6a8077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- 172.21.15.37:0/2774652536 shutdown_connections 2024-04-25T21:15:16.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- 172.21.15.37:0/2774652536 wait complete. 2024-04-25T21:15:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 Processor -- start 2024-04-25T21:15:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.194+0000 7fd6aef44700 1 -- start start 2024-04-25T21:15:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 0x7fd6a8101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 0x7fd6a8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112c90 con 0x7fd6a806bb70 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112b10 con 0x7fd6a8108a50 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6aef44700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a8112e10 con 0x7fd6a806b1b0 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54568/0 (socket says 172.21.15.37:54568) 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 -- 172.21.15.37:0/4053359948 learned_addr learned my addr 172.21.15.37:0/4053359948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 0x7fd6a8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6acce0700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 0x7fd6a8101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 -- 172.21.15.37:0/4053359948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 msgr2=0x7fd6a8101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 0x7fd6a8101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 -- 172.21.15.37:0/4053359948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 msgr2=0x7fd6a8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 0x7fd6a8102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 -- 172.21.15.37:0/4053359948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a81a7550 con 0x7fd6a806bb70 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6acce0700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 0x7fd6a8101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd6940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.195+0000 7fd6a5ffb700 1 -- 172.21.15.37:0/4053359948 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694027070 con 0x7fd6a806bb70 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.196+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a81a77e0 con 0x7fd6a806bb70 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.196+0000 7fd6a5ffb700 1 -- 172.21.15.37:0/4053359948 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd694003f20 con 0x7fd6a806bb70 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.196+0000 7fd6a5ffb700 1 -- 172.21.15.37:0/4053359948 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694022cf0 con 0x7fd6a806bb70 2024-04-25T21:15:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.196+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a81a7c40 con 0x7fd6a806bb70 2024-04-25T21:15:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.198+0000 7fd6a5ffb700 1 -- 172.21.15.37:0/4053359948 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd69402a070 con 0x7fd6a806bb70 2024-04-25T21:15:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.198+0000 7fd6a5ffb700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 0x7fd6980a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6a5ffb700 1 -- 172.21.15.37:0/4053359948 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6940d15a0 con 0x7fd6a806bb70 2024-04-25T21:15:16.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6acce0700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 0x7fd6980a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6acce0700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 0x7fd6980a6310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005fd0 tx=0x7fd69c005db0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 Processor -- start 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 -- start start 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708eee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557e70841ec0 con 0x557e708fa270 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e707e8af0 con 0x557e708ef3a0 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.199+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557e7083de80 con 0x557e708eea80 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54584/0 (socket says 172.21.15.37:54584) 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3361929442 learned_addr learned my addr 172.21.15.37:0/3361929442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:16.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3361929442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 msgr2=0x557e708eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708eee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3361929442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 msgr2=0x557e708f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3361929442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e7078fa60 con 0x557e708fa270 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098f0 tx=0x7fd6a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842163f547cf96e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd692ffd700 1 -- 172.21.15.37:0/3361929442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004bb0 con 0x557e708fa270 2024-04-25T21:15:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd692ffd700 1 -- 172.21.15.37:0/3361929442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a0004d50 con 0x557e708fa270 2024-04-25T21:15:16.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.200+0000 7fd692ffd700 1 -- 172.21.15.37:0/3361929442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0007e40 con 0x557e708fa270 2024-04-25T21:15:16.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3361929442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 msgr2=0x557e708fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098f0 tx=0x7fd6a0009c00 comp rx=0 tx=0).stop 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3361929442 shutdown_connections 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fc660 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3361929442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3361929442 >> 172.21.15.37:0/3361929442 conn(0x557e7081c820 msgr2=0x557e708eaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3361929442 shutdown_connections 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3361929442 wait complete. 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.201+0000 7fd6b97cdb80 1 -- start start 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 Processor -- start 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708f9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557e708fcc60 con 0x557e708fa270 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557e707e0d60 con 0x557e708ef3a0 2024-04-25T21:15:16.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557e708fcde0 con 0x557e708eea80 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6b97cdb80 10 client.0 _ll_register_callbacks cb 0x557e707daae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6ad4e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6ad4e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54592/0 (socket says 172.21.15.37:54592) 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3856384013 learned_addr learned my addr 172.21.15.37:0/3856384013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.202+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 msgr2=0x557e708f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708f9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3856384013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 msgr2=0x557e708f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/3856384013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e707f9400 con 0x557e708fa270 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:16.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0007a60 tx=0x7fd6a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf91865ed7f9a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0022070 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6a0004d50 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6a0007cd0 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6a002e050 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6b97cdb80 10 client.16173 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e709073e0 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a0004d50 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6a001e4a0 con 0x557e708fa270 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6b97cdb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.203+0000 7fd6b97cdb80 10 client.16173 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6b97cdb80 20 client.16173 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70909910 0x557e7090bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6b97cdb80 4 client.16173 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:16.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e7084f6b0 con 0x557e70909910 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70909910 0x557e7090bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd6acce0700 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70909910 0x557e7090bd00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293304f8844b46e4 server_cookie=9a617df403a4ae6e in_seq=0 out_seq=0 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.204+0000 7fd687fff700 10 client.16173 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd687fff700 1 -- 172.21.15.37:0/3856384013 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e7084f6b0 con 0x557e70909910 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd687fff700 10 client.16173 handle_command_reply: tid=0 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 client.16173 shutdown 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 msgr2=0x557e708f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0007a60 tx=0x7fd6a00046b0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 shutdown_connections 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70909910 0x557e7090bd00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708f71c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e708f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/3856384013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 >> 172.21.15.37:0/3856384013 conn(0x557e7081c820 msgr2=0x557e708eab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.205+0000 7fd687fff700 0 client.16173 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:16.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.206+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 shutdown_connections 2024-04-25T21:15:16.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.206+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/3856384013 wait complete. 2024-04-25T21:15:16.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.206+0000 7fd6b97cdb80 20 client.16173 trim_cache size 0 max 16384 2024-04-25T21:15:16.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 Processor -- start 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 -- start start 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e70911330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708fa270 0x557e70911870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81af230 con 0x557e708eea80 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81b0d70 con 0x557e708ef3a0 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.225+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81af900 con 0x557e708fa270 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54606/0 (socket says 172.21.15.37:54606) 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 -- 172.21.15.37:0/1349089863 learned_addr learned my addr 172.21.15.37:0/1349089863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e70911330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708fa270 0x557e70911870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 -- 172.21.15.37:0/1349089863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708fa270 msgr2=0x557e70911870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708fa270 0x557e70911870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 -- 172.21.15.37:0/1349089863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 msgr2=0x557e70911330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e70911330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 -- 172.21.15.37:0/1349089863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e708f0200 con 0x557e708eea80 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.226+0000 7fd6acce0700 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005950 tx=0x7fd69c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf334e2685d7ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6927fc700 1 -- 172.21.15.37:0/1349089863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c020c70 con 0x557e708eea80 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6927fc700 1 -- 172.21.15.37:0/1349089863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c020e10 con 0x557e708eea80 2024-04-25T21:15:16.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6927fc700 1 -- 172.21.15.37:0/1349089863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c005020 con 0x557e708eea80 2024-04-25T21:15:16.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1349089863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 msgr2=0x557e708eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005950 tx=0x7fd69c014000 comp rx=0 tx=0).stop 2024-04-25T21:15:16.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1349089863 shutdown_connections 2024-04-25T21:15:16.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708fa270 0x557e70911870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708eea80 0x557e708eb050 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1349089863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e70911330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1349089863 >> 172.21.15.37:0/1349089863 conn(0x557e7081c820 msgr2=0x557e708f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1349089863 shutdown_connections 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.227+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1349089863 wait complete. 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 -- start start 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 Processor -- start 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e70902340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708fe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81b06a0 con 0x557e708fa270 2024-04-25T21:15:16.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81affd0 con 0x557e708ef3a0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a81ad1f0 con 0x557e708eea80 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6b97cdb80 10 client.0 _ll_register_callbacks cb 0x557e707daae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6acce0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e70902340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6ad4e1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6acce0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e70902340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:46918/0 (socket says 172.21.15.37:46918) 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6acce0700 1 -- 172.21.15.37:0/1673531974 learned_addr learned my addr 172.21.15.37:0/1673531974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.228+0000 7fd6a7fff700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708fe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 msgr2=0x557e70902340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e70902340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/1673531974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 msgr2=0x557e708fe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708fe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 -- 172.21.15.37:0/1673531974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e708ff4a0 con 0x557e708fa270 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6ad4e1700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fec60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00093f0 tx=0x7fd6a0009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b483e5aa2ca9277e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd687fff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:16.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0022040 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd69c020e10 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6a001a040 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6a0007cd0 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6b97cdb80 10 client.16179 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557e70970b20 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.229+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd69c020e10 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6a001eab0 con 0x557e708fa270 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 10 client.16179 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 10 client.16179 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 20 client.16179 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70971190 0x557e70973580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 4 client.16179 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}] 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}) -- 0x557e70849750 con 0x557e70971190 2024-04-25T21:15:16.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.230+0000 7fd6acce0700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70971190 0x557e70973580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.231+0000 7fd6acce0700 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70971190 0x557e70973580 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84fa76b89b290ec7 server_cookie=2dd9371f67ffbfef in_seq=0 out_seq=0 2024-04-25T21:15:16.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.231+0000 7fd687fff700 10 client.16179 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd687fff700 1 -- 172.21.15.37:0/1673531974 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557e70849750 con 0x557e70971190 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd687fff700 10 client.16179 handle_command_reply: tid=0 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 client.16179 shutdown 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 msgr2=0x557e708fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fec60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00093f0 tx=0x7fd6a0009100 comp rx=0 tx=0).stop 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 shutdown_connections 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557e70971190 0x557e70973580 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557e708fa270 0x557e708fec60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e708eea80 0x557e70902340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 --2- 172.21.15.37:0/1673531974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557e708ef3a0 0x557e708fe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.245+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 >> 172.21.15.37:0/1673531974 conn(0x557e7081c820 msgr2=0x557e708f84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd687fff700 0 client.16179 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 shutdown_connections 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6b97cdb80 1 -- 172.21.15.37:0/1673531974 wait complete. 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6b97cdb80 20 client.16179 trim_cache size 0 max 16384 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:15:16.247 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 18, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "8802b09a": { 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 18, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.8, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:15:16.248 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:16.249 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 msgr2=0x7fd6980a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 0x7fd6980a6310 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd69c005fd0 tx=0x7fd69c005db0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 msgr2=0x7fd6a8101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.246+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd694000c00 tx=0x7fd6940047b0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 shutdown_connections 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6980a3e50 0x7fd6980a6310 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6a8108a50 0x7fd6a8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6a806b1b0 0x7fd6a8101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 --2- 172.21.15.37:0/4053359948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6a806bb70 0x7fd6a8101fe0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 >> 172.21.15.37:0/4053359948 conn(0x7fd6a8077990 msgr2=0x7fd6a80ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 shutdown_connections 2024-04-25T21:15:16.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:16.247+0000 7fd6aef44700 1 -- 172.21.15.37:0/4053359948 wait complete. 2024-04-25T21:15:16.978 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8802b09a', quiesce took 6.1 seconds. Will release after: 29.8 2024-04-25T21:15:17.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:17 smithi037 ceph-mon[29195]: pgmap v209: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:15:17.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:17 smithi177 ceph-mon[33043]: pgmap v209: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:15:17.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:17 smithi098 ceph-mon[32569]: pgmap v209: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/1775833829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f8101aa0 msgr2=0x7fd6f8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/1775833829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f8101aa0 0x7fd6f8105a90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e002a410 comp rx=0 tx=0).stop 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/1775833829 shutdown_connections 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/1775833829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 0x7fd6f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/1775833829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f8101aa0 0x7fd6f8105a90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/1775833829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f81010f0 0x7fd6f81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/1775833829 >> 172.21.15.37:0/1775833829 conn(0x7fd6f8077fd0 msgr2=0x7fd6f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/1775833829 shutdown_connections 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/1775833829 wait complete. 2024-04-25T21:15:18.319 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 Processor -- start 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 -- start start 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.317+0000 7fd6ff1f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 0x7fd6f8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f8199fc0 0x7fd6f81a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81110b0 con 0x7fd6f81010f0 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8110f30 con 0x7fd6f8106170 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8111230 con 0x7fd6f8199fc0 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54644/0 (socket says 172.21.15.37:54644) 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/2083770645 learned_addr learned my addr 172.21.15.37:0/2083770645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f8199fc0 0x7fd6f81a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 0x7fd6f8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/2083770645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f8199fc0 msgr2=0x7fd6f81a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f8199fc0 0x7fd6f81a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/2083770645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 msgr2=0x7fd6f8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 0x7fd6f8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/2083770645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f81a4d70 con 0x7fd6f81010f0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e9f0 tx=0x7fd6e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6f5ffb640 1 -- 172.21.15.37:0/2083770645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800cd80 con 0x7fd6f81010f0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6f5ffb640 1 -- 172.21.15.37:0/2083770645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e80054d0 con 0x7fd6f81010f0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f81a5060 con 0x7fd6f81010f0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.318+0000 7fd6f5ffb640 1 -- 172.21.15.37:0/2083770645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8010690 con 0x7fd6f81010f0 2024-04-25T21:15:18.321 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.319+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f81a5530 con 0x7fd6f81010f0 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.321+0000 7fd6f5ffb640 1 -- 172.21.15.37:0/2083770645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6e8004e90 con 0x7fd6f81010f0 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.321+0000 7fd701fde740 1 Processor -- start 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.321+0000 7fd701fde740 1 -- start start 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.321+0000 7fd6f5ffb640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 0x7fd6e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe974a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe981bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563bbe8d9970 con 0x563bbe974f40 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563bbe8764a0 con 0x563bbe97f800 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd701fde740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563bbe8d5930 con 0x563bbe974620 2024-04-25T21:15:18.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 0x7fd6e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f5ffb640 1 -- 172.21.15.37:0/2083770645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6e80d0c60 con 0x7fd6f81010f0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54660/0 (socket says 172.21.15.37:54660) 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3612644430 learned_addr learned my addr 172.21.15.37:0/3612644430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe974a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3612644430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 msgr2=0x563bbe974a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe974a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe981bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3612644430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 msgr2=0x563bbe981bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe981bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 0x7fd6e40a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005fd0 tx=0x7fd6f00107a0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3612644430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbe5ec210 con 0x563bbe974f40 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e0032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754d8a20961ce00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.322+0000 7fd6eeffd640 1 -- 172.21.15.37:0/3612644430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0040070 con 0x563bbe974f40 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd6eeffd640 1 -- 172.21.15.37:0/3612644430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0004030 con 0x563bbe974f40 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd6eeffd640 1 -- 172.21.15.37:0/3612644430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035040 con 0x563bbe974f40 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 -- 172.21.15.37:0/3612644430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 msgr2=0x563bbe97f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e0032b20 comp rx=0 tx=0).stop 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 -- 172.21.15.37:0/3612644430 shutdown_connections 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe981bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97f2c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 --2- 172.21.15.37:0/3612644430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 -- 172.21.15.37:0/3612644430 >> 172.21.15.37:0/3612644430 conn(0x563bbe970b30 msgr2=0x563bbe970f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 -- 172.21.15.37:0/3612644430 shutdown_connections 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.323+0000 7fd701fde740 1 -- 172.21.15.37:0/3612644430 wait complete. 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 -- start start 2024-04-25T21:15:18.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 Processor -- start 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 0x563bbe97c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe97f800 0x563bbe97ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563bbe9823d0 con 0x563bbe974620 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563bbe8740b0 con 0x563bbe974f40 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563bbe982550 con 0x563bbe97f800 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd701fde740 10 client.0 _ll_register_callbacks cb 0x563bbe861d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd6fcf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd6fcf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54662/0 (socket says 172.21.15.37:54662) 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.324+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/942698717 learned_addr learned my addr 172.21.15.37:0/942698717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe97f800 0x563bbe97ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 0x563bbe97c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe97f800 msgr2=0x563bbe97ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe97f800 0x563bbe97ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/942698717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 msgr2=0x563bbe97c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 0x563bbe97c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 -- 172.21.15.37:0/942698717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbe6b8620 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 0x563bbe97c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6e809c8a0 tx=0x7fd6e8005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1431b5d4466aa34b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80a0f50 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6e80bd0c0 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e0004030 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6e80a5810 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd701fde740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bbe98e320 con 0x563bbe974620 2024-04-25T21:15:18.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.325+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0004030 con 0x563bbe974620 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e80988f0 con 0x563bbe974620 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 20 client.16191 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe98f7e0 0x563bbe991bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 4 client.16191 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bbe8e7390 con 0x563bbe98f7e0 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.326+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe98f7e0 0x563bbe991bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe98f7e0 0x563bbe991bd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a2c776831f71556 server_cookie=909c45b1c4d9e3a in_seq=0 out_seq=0 2024-04-25T21:15:18.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd6d7fff640 10 client.16191 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd6d7fff640 1 -- 172.21.15.37:0/942698717 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bbe8e7390 con 0x563bbe98f7e0 2024-04-25T21:15:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd6d7fff640 10 client.16191 handle_command_reply: tid=0 2024-04-25T21:15:18.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd701fde740 1 client.16191 shutdown 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 msgr2=0x563bbe97f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd6e809c8a0 tx=0x7fd6e8005180 comp rx=0 tx=0).stop 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 shutdown_connections 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.327+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe98f7e0 0x563bbe991bd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe97f800 0x563bbe97ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe974f40 0x563bbe97c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd6d7fff640 0 client.16191 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 --2- 172.21.15.37:0/942698717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974620 0x563bbe97f190 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 >> 172.21.15.37:0/942698717 conn(0x563bbe970b30 msgr2=0x563bbe976860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 shutdown_connections 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 1 -- 172.21.15.37:0/942698717 wait complete. 2024-04-25T21:15:18.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.328+0000 7fd701fde740 20 client.16191 trim_cache size 0 max 16384 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 Processor -- start 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 -- start start 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe98b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe9974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81adab0 con 0x563bbe974f40 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81af5f0 con 0x563bbe97f800 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd701fde740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81ae180 con 0x563bbe974620 2024-04-25T21:15:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54664/0 (socket says 172.21.15.37:54664) 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 -- 172.21.15.37:0/554045444 learned_addr learned my addr 172.21.15.37:0/554045444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe9974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe98b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 -- 172.21.15.37:0/554045444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 msgr2=0x563bbe98b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe98b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 -- 172.21.15.37:0/554045444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 msgr2=0x563bbe9974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe9974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 -- 172.21.15.37:0/554045444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bbe97cd70 con 0x563bbe974f40 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe9974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.343+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00096c0 tx=0x7fd6e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd34f7554be41514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd6ee7fc640 1 -- 172.21.15.37:0/554045444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0040070 con 0x563bbe974f40 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd6ee7fc640 1 -- 172.21.15.37:0/554045444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0005040 con 0x563bbe974f40 2024-04-25T21:15:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd6ee7fc640 1 -- 172.21.15.37:0/554045444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035040 con 0x563bbe974f40 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 -- 172.21.15.37:0/554045444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 msgr2=0x563bbe97ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00096c0 tx=0x7fd6e002a410 comp rx=0 tx=0).stop 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 -- 172.21.15.37:0/554045444 shutdown_connections 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe9974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe97ecc0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 --2- 172.21.15.37:0/554045444 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe98b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 -- 172.21.15.37:0/554045444 >> 172.21.15.37:0/554045444 conn(0x563bbe970b30 msgr2=0x563bbe9779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 -- 172.21.15.37:0/554045444 shutdown_connections 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.344+0000 7fd701fde740 1 -- 172.21.15.37:0/554045444 wait complete. 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 -- start start 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 Processor -- start 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe987990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe979250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81aef20 con 0x563bbe974f40 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81ae850 con 0x563bbe97f800 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81abab0 con 0x563bbe974620 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd701fde740 10 client.0 _ll_register_callbacks cb 0x7fd6e809c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd6f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54678/0 (socket says 172.21.15.37:54678) 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3678853373 learned_addr learned my addr 172.21.15.37:0/3678853373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe987990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.345+0000 7fd6fd78d640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe979250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 msgr2=0x563bbe987990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe987990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3678853373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 msgr2=0x563bbe979250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe979250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 -- 172.21.15.37:0/3678853373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bbe979890 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6f7fff640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0031440 tx=0x7fd6e0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc384b4fd6dac89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035070 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e0005040 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fd6e0005400 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd6e0040040 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd701fde740 10 client.16197 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.346+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563bbe9f5bd0 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0005040 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd6e0004ac0 con 0x563bbe974f40 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 10 client.16197 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 20 client.16197 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe9f5e50 0x563bbe9f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 4 client.16197 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bbe8e1200 con 0x563bbe9f5e50 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe9f5e50 0x563bbe9f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd6d7fff640 10 client.16197 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.347+0000 7fd6fcf8c640 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe9f5e50 0x563bbe9f8240 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf56efd8cc12f7a server_cookie=6d71b6fbd383915a in_seq=0 out_seq=0 2024-04-25T21:15:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd6d7fff640 1 -- 172.21.15.37:0/3678853373 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bbe8e1200 con 0x563bbe9f5e50 2024-04-25T21:15:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd6d7fff640 10 client.16197 handle_command_reply: tid=0 2024-04-25T21:15:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 client.16197 shutdown 2024-04-25T21:15:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 msgr2=0x563bbe978d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0031440 tx=0x7fd6e0004410 comp rx=0 tx=0).stop 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 shutdown_connections 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563bbe9f5e50 0x563bbe9f8240 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563bbe97f800 0x563bbe979250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563bbe974f40 0x563bbe978d10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 --2- 172.21.15.37:0/3678853373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563bbe974620 0x563bbe987990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd6d7fff640 0 client.16197 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 >> 172.21.15.37:0/3678853373 conn(0x563bbe970b30 msgr2=0x563bbe975fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 shutdown_connections 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 1 -- 172.21.15.37:0/3678853373 wait complete. 2024-04-25T21:15:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.364+0000 7fd701fde740 20 client.16197 trim_cache size 0 max 16384 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 msgr2=0x7fd6e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 0x7fd6e40a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0005fd0 tx=0x7fd6f00107a0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 msgr2=0x7fd6f81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800e9f0 tx=0x7fd6e800edf0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 shutdown_connections 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd6e40a3dc0 0x7fd6e40a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f8199fc0 0x7fd6f81a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6f8106170 0x7fd6f8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 --2- 172.21.15.37:0/2083770645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6f81010f0 0x7fd6f81994f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 >> 172.21.15.37:0/2083770645 conn(0x7fd6f8077fd0 msgr2=0x7fd6f80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 shutdown_connections 2024-04-25T21:15:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:18.365+0000 7fd6ff1f0640 1 -- 172.21.15.37:0/2083770645 wait complete. 2024-04-25T21:15:19.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:19 smithi037 ceph-mon[29195]: pgmap v210: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:15:19.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:19 smithi177 ceph-mon[33043]: pgmap v210: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:15:19.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:19 smithi098 ceph-mon[32569]: pgmap v210: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:15:21.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:21 smithi037 ceph-mon[29195]: pgmap v211: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:15:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:21 smithi177 ceph-mon[33043]: pgmap v211: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:15:21.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:21 smithi098 ceph-mon[32569]: pgmap v211: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-25T21:15:23.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:23 smithi037 ceph-mon[29195]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-25T21:15:23.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:23 smithi177 ceph-mon[33043]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-25T21:15:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:23 smithi098 ceph-mon[32569]: pgmap v212: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-25T21:15:25.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:25 smithi037 ceph-mon[29195]: pgmap v213: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-25T21:15:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:25 smithi177 ceph-mon[33043]: pgmap v213: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-25T21:15:25.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:25 smithi098 ceph-mon[32569]: pgmap v213: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-25T21:15:27.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:27 smithi037 ceph-mon[29195]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:27 smithi177 ceph-mon[33043]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:27 smithi098 ceph-mon[32569]: pgmap v214: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:29.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:29 smithi037 ceph-mon[29195]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:29.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:29 smithi177 ceph-mon[33043]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:29 smithi098 ceph-mon[32569]: pgmap v215: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-25T21:15:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 -- 172.21.15.37:0/2266734222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a81021c0 msgr2=0x7f50a81025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 --2- 172.21.15.37:0/2266734222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a81021c0 0x7f50a81025a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5098009900 tx=0x7f509802a410 comp rx=0 tx=0).stop 2024-04-25T21:15:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 -- 172.21.15.37:0/2266734222 shutdown_connections 2024-04-25T21:15:31.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 --2- 172.21.15.37:0/2266734222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 0x7f50a810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 --2- 172.21.15.37:0/2266734222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a8102b70 0x7f50a8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 --2- 172.21.15.37:0/2266734222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a81021c0 0x7f50a81025a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.466+0000 7f50ac872640 1 -- 172.21.15.37:0/2266734222 >> 172.21.15.37:0/2266734222 conn(0x7f50a80fda20 msgr2=0x7f50a80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.467+0000 7f50ac872640 1 -- 172.21.15.37:0/2266734222 shutdown_connections 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.467+0000 7f50ac872640 1 -- 172.21.15.37:0/2266734222 wait complete. 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.467+0000 7f50ac872640 1 Processor -- start 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.467+0000 7f50ac872640 1 -- start start 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a81021c0 0x7f50a81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 0x7f50a81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112200 con 0x7f50a819a740 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112080 con 0x7f50a81021c0 2024-04-25T21:15:31.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50ac872640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8112380 con 0x7f50a8107290 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37088/0 (socket says 172.21.15.37:37088) 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 -- 172.21.15.37:0/1312512039 learned_addr learned my addr 172.21.15.37:0/1312512039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a659c640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a81021c0 0x7f50a81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f509ffff640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 0x7f50a81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 -- 172.21.15.37:0/1312512039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 msgr2=0x7f50a81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 0x7f50a81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 -- 172.21.15.37:0/1312512039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a81021c0 msgr2=0x7f50a81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a81021c0 0x7f50a81a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 -- 172.21.15.37:0/1312512039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a819b340 con 0x7f50a819a740 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.468+0000 7f50a6d9d640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f50a000eaa0 tx=0x7f50a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.469+0000 7f509f7fe640 1 -- 172.21.15.37:0/1312512039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a000cdb0 con 0x7f50a819a740 2024-04-25T21:15:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.469+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a81a7420 con 0x7f50a819a740 2024-04-25T21:15:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.469+0000 7f509f7fe640 1 -- 172.21.15.37:0/1312512039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0005020 con 0x7f50a819a740 2024-04-25T21:15:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.469+0000 7f509f7fe640 1 -- 172.21.15.37:0/1312512039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00116d0 con 0x7f50a819a740 2024-04-25T21:15:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.469+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81a7820 con 0x7f50a819a740 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 Processor -- start 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f509f7fe640 1 -- 172.21.15.37:0/1312512039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50a00054a0 con 0x7f50a819a740 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 -- start start 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf07bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5613bde5f970 con 0x5613bdefa620 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5613bddfc4a0 con 0x5613bdefaf40 2024-04-25T21:15:31.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.472+0000 7f50af588740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613bde5b930 con 0x5613bdf05800 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37098/0 (socket says 172.21.15.37:37098) 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 -- 172.21.15.37:0/3834236610 learned_addr learned my addr 172.21.15.37:0/3834236610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509f7fe640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 0x7f50940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f50a659c640 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf07bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f50a6d9d640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 0x7f50940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 -- 172.21.15.37:0/3834236610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 msgr2=0x5613bdf07bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf07bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 -- 172.21.15.37:0/3834236610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 msgr2=0x5613bdf052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509f7fe640 1 -- 172.21.15.37:0/1312512039 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f50a00d0930 con 0x7f50a819a740 2024-04-25T21:15:31.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 -- 172.21.15.37:0/3834236610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bdb72210 con 0x5613bdefa620 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509ffff640 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f509000dfa0 tx=0x7f509000d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a56594dccd0d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f50a6d9d640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 0x7f50940a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a00ce130 tx=0x7f50a001eb40 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509cff9640 1 -- 172.21.15.37:0/3834236610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090004030 con 0x5613bdefa620 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509cff9640 1 -- 172.21.15.37:0/3834236610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50900041d0 con 0x5613bdefa620 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.473+0000 7f509cff9640 1 -- 172.21.15.37:0/3834236610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090011640 con 0x5613bdefa620 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 -- 172.21.15.37:0/3834236610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 msgr2=0x5613bdefaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f509000dfa0 tx=0x7f509000d650 comp rx=0 tx=0).stop 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 -- 172.21.15.37:0/3834236610 shutdown_connections 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf07bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 --2- 172.21.15.37:0/3834236610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdefaa00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 -- 172.21.15.37:0/3834236610 >> 172.21.15.37:0/3834236610 conn(0x5613bdef6b30 msgr2=0x5613bdef6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 -- 172.21.15.37:0/3834236610 shutdown_connections 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.474+0000 7f50af588740 1 -- 172.21.15.37:0/3834236610 wait complete. 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 -- start start 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 Processor -- start 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdefa620 0x5613bdf050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf023e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5613bdf08430 con 0x5613bdf05800 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5613bddfa0b0 con 0x5613bdefaf40 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5613bdf085b0 con 0x5613bdefa620 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50af588740 10 client.0 _ll_register_callbacks cb 0x5613bdde7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37102/0 (socket says 172.21.15.37:37102) 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 -- 172.21.15.37:0/1727062179 learned_addr learned my addr 172.21.15.37:0/1727062179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdefa620 0x5613bdf050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f50a659c640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 -- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdefa620 msgr2=0x5613bdf050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdefa620 0x5613bdf050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 -- 172.21.15.37:0/1727062179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 msgr2=0x5613bdf023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.475+0000 7f509ffff640 1 -- 172.21.15.37:0/1727062179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bdc3e620 con 0x5613bdf05800 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f509ffff640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f509000d620 tx=0x7f509000d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b312641e2ee0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090022070 con 0x5613bdf05800 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50900041d0 con 0x5613bdf05800 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f509001a030 con 0x5613bdf05800 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5090034050 con 0x5613bdf05800 2024-04-25T21:15:31.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 10 client.16209 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613bdf14340 con 0x5613bdf05800 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50900041d0 con 0x5613bdf05800 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f50900046c0 con 0x5613bdf05800 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 20 client.16209 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.476+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf15740 0x5613bdf17b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.477+0000 7f50af588740 4 client.16209 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.477+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613bde6d390 con 0x5613bdf15740 2024-04-25T21:15:31.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.477+0000 7f50a659c640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf15740 0x5613bdf17b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.477+0000 7f50a659c640 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf15740 0x5613bdf17b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fd5dcd6e40afd7 server_cookie=38944662bae1bef4 in_seq=0 out_seq=0 2024-04-25T21:15:31.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.477+0000 7f508dffb640 10 client.16209 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f508dffb640 1 -- 172.21.15.37:0/1727062179 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613bde6d390 con 0x5613bdf15740 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f508dffb640 10 client.16209 handle_command_reply: tid=0 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 client.16209 shutdown 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 msgr2=0x5613bdf02920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f509000d620 tx=0x7f509000d5b0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 shutdown_connections 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf15740 0x5613bdf17b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdf05800 0x5613bdf02920 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 --2- 172.21.15.37:0/1727062179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdefa620 0x5613bdf050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f508dffb640 0 client.16209 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.478+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 >> 172.21.15.37:0/1727062179 conn(0x5613bdef6b30 msgr2=0x5613bdefc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.479+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 shutdown_connections 2024-04-25T21:15:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.479+0000 7f50af588740 1 -- 172.21.15.37:0/1727062179 wait complete. 2024-04-25T21:15:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.479+0000 7f50af588740 20 client.16209 trim_cache size 0 max 16384 2024-04-25T21:15:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.493+0000 7f50af588740 1 Processor -- start 2024-04-25T21:15:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 -- start start 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf04c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf1d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81afd20 con 0x5613bdefa620 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81b1860 con 0x5613bdefaf40 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f50af588740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81b03f0 con 0x5613bdf05800 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37114/0 (socket says 172.21.15.37:37114) 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 -- 172.21.15.37:0/3560738189 learned_addr learned my addr 172.21.15.37:0/3560738189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 -- 172.21.15.37:0/3560738189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 msgr2=0x5613bdf1d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf1d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 -- 172.21.15.37:0/3560738189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 msgr2=0x5613bdf04c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf04c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 -- 172.21.15.37:0/3560738189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bdf02f20 con 0x5613bdefa620 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f509ffff640 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f509000d6d0 tx=0x7f509000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d9ca8483991f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.494+0000 7f508ffff640 1 -- 172.21.15.37:0/3560738189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509003d070 con 0x5613bdefa620 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f508ffff640 1 -- 172.21.15.37:0/3560738189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50900040c0 con 0x5613bdefa620 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f508ffff640 1 -- 172.21.15.37:0/3560738189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090022040 con 0x5613bdefa620 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- 172.21.15.37:0/3560738189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 msgr2=0x5613bdf112f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f509000d6d0 tx=0x7f509000d890 comp rx=0 tx=0).stop 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- 172.21.15.37:0/3560738189 shutdown_connections 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf1d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefaf40 0x5613bdf04c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 --2- 172.21.15.37:0/3560738189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefa620 0x5613bdf112f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- 172.21.15.37:0/3560738189 >> 172.21.15.37:0/3560738189 conn(0x5613bdef6b30 msgr2=0x5613bdf01410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- 172.21.15.37:0/3560738189 shutdown_connections 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- 172.21.15.37:0/3560738189 wait complete. 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.495+0000 7f50af588740 1 -- start start 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 Processor -- start 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefa620 0x5613bdf09300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81b1190 con 0x5613bdefaf40 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81b0ac0 con 0x5613bdefa620 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81add20 con 0x5613bdf05800 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 10 client.0 _ll_register_callbacks cb 0x7f5090003f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37118/0 (socket says 172.21.15.37:37118) 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 -- 172.21.15.37:0/2710697450 learned_addr learned my addr 172.21.15.37:0/2710697450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:31.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f509ffff640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefa620 0x5613bdf09300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 -- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 msgr2=0x5613bdf09d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 -- 172.21.15.37:0/2710697450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefa620 msgr2=0x5613bdf09300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefa620 0x5613bdf09300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 -- 172.21.15.37:0/2710697450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bdf7aa70 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50a659c640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5098009900 tx=0x7f5098009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4730542f4f0db234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f508dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509804a060 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f5098045bc0 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.496+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50900040c0 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5098035040 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 10 client.16215 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613bdf7bd80 con 0x5613bdefaf40 2024-04-25T21:15:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50900040c0 con 0x5613bdefaf40 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f50980053c0 con 0x5613bdefaf40 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 20 client.16215 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf7c480 0x5613bdf7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 4 client.16215 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613bde67200 con 0x5613bdf7c480 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.497+0000 7f509ffff640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf7c480 0x5613bdf7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.498+0000 7f508dffb640 10 client.16215 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.498+0000 7f509ffff640 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf7c480 0x5613bdf7e8a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae533517783b1f4c server_cookie=20e5fa0cb16f5228 in_seq=0 out_seq=0 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f508dffb640 1 -- 172.21.15.37:0/2710697450 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613bde67200 con 0x5613bdf7c480 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f508dffb640 10 client.16215 handle_command_reply: tid=0 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 client.16215 shutdown 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 msgr2=0x5613bdf09840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5098009900 tx=0x7f5098009660 comp rx=0 tx=0).stop 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 shutdown_connections 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5613bdf7c480 0x5613bdf7e8a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5613bdf05800 0x5613bdf09d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5613bdefaf40 0x5613bdf09840 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f508dffb640 0 client.16215 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.511+0000 7f50af588740 1 --2- 172.21.15.37:0/2710697450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5613bdefa620 0x5613bdf09300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 >> 172.21.15.37:0/2710697450 conn(0x5613bdef6b30 msgr2=0x5613bdefbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 shutdown_connections 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50af588740 1 -- 172.21.15.37:0/2710697450 wait complete. 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50af588740 20 client.16215 trim_cache size 0 max 16384 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:15:31.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 msgr2=0x7f50940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 0x7f50940a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f50a00ce130 tx=0x7f50a001eb40 comp rx=0 tx=0).stop 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 msgr2=0x7f50a819abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f50a000eaa0 tx=0x7f50a000ef60 comp rx=0 tx=0).stop 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 shutdown_connections 2024-04-25T21:15:31.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50a819a740 0x7f50a819abc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f50940a3dc0 0x7f50940a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f50a8107290 0x7f50a81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 --2- 172.21.15.37:0/1312512039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50a81021c0 0x7f50a81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 >> 172.21.15.37:0/1312512039 conn(0x7f50a80fda20 msgr2=0x7f50a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 shutdown_connections 2024-04-25T21:15:31.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:31.512+0000 7f50ac872640 1 -- 172.21.15.37:0/1312512039 wait complete. 2024-04-25T21:15:31.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:31 smithi037 ceph-mon[29195]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T21:15:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:31 smithi177 ceph-mon[33043]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T21:15:31.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:31 smithi098 ceph-mon[32569]: pgmap v216: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-25T21:15:33.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:33 smithi037 ceph-mon[29195]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:15:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:33 smithi177 ceph-mon[33043]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:15:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:33 smithi098 ceph-mon[32569]: pgmap v217: 97 pgs: 97 active+clean; 4.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T21:15:35.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:35 smithi037 ceph-mon[29195]: pgmap v218: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T21:15:35.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:35 smithi177 ceph-mon[33043]: pgmap v218: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T21:15:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:35 smithi098 ceph-mon[32569]: pgmap v218: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-25T21:15:37.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:37 smithi037 ceph-mon[29195]: pgmap v219: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:37 smithi177 ceph-mon[33043]: pgmap v219: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:37.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:37 smithi098 ceph-mon[32569]: pgmap v219: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:39 smithi037 ceph-mon[29195]: pgmap v220: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:39 smithi177 ceph-mon[33043]: pgmap v220: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:39.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:39 smithi098 ceph-mon[32569]: pgmap v220: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-25T21:15:40.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:40 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:15:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:40 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:15:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:40 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:15:41.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:41 smithi037 ceph-mon[29195]: pgmap v221: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-04-25T21:15:41.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:15:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:15:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:15:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:41 smithi177 ceph-mon[33043]: pgmap v221: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-04-25T21:15:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:15:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:15:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:15:41.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:41 smithi098 ceph-mon[32569]: pgmap v221: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 14 op/s 2024-04-25T21:15:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:15:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:15:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:15:43.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:43 smithi037 ceph-mon[29195]: pgmap v222: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T21:15:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:43 smithi177 ceph-mon[33043]: pgmap v222: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T21:15:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:43 smithi098 ceph-mon[32569]: pgmap v222: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-25T21:15:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 -- 172.21.15.37:0/3560490284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 msgr2=0x7f6e5c10e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 --2- 172.21.15.37:0/3560490284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c10e390 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6e44009900 tx=0x7f6e4402a410 comp rx=0 tx=0).stop 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 -- 172.21.15.37:0/3560490284 shutdown_connections 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 --2- 172.21.15.37:0/3560490284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 0x7f6e5c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 --2- 172.21.15.37:0/3560490284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c10e390 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 --2- 172.21.15.37:0/3560490284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 0x7f6e5c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 -- 172.21.15.37:0/3560490284 >> 172.21.15.37:0/3560490284 conn(0x7f6e5c077fd0 msgr2=0x7f6e5c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 -- 172.21.15.37:0/3560490284 shutdown_connections 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 -- 172.21.15.37:0/3560490284 wait complete. 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.612+0000 7f6e6215d640 1 Processor -- start 2024-04-25T21:15:44.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 -- start start 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 0x7f6e5c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 0x7f6e5c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c114410 con 0x7f6e5c1016d0 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c114290 con 0x7f6e5c100d20 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e6215d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c114590 con 0x7f6e5c10e8d0 2024-04-25T21:15:44.615 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60646/0 (socket says 172.21.15.37:60646) 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 -- 172.21.15.37:0/1461305559 learned_addr learned my addr 172.21.15.37:0/1461305559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 0x7f6e5c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 0x7f6e5c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 -- 172.21.15.37:0/1461305559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 msgr2=0x7f6e5c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 0x7f6e5c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 -- 172.21.15.37:0/1461305559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 msgr2=0x7f6e5c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 0x7f6e5c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.613+0000 7f6e5affd640 1 -- 172.21.15.37:0/1461305559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c19b030 con 0x7f6e5c1016d0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e5affd640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e44005940 tx=0x7f6e440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e58ff9640 1 -- 172.21.15.37:0/1461305559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4403f030 con 0x7f6e5c1016d0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5c19b2c0 con 0x7f6e5c1016d0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e58ff9640 1 -- 172.21.15.37:0/1461305559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e44003cb0 con 0x7f6e5c1016d0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e58ff9640 1 -- 172.21.15.37:0/1461305559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44035070 con 0x7f6e5c1016d0 2024-04-25T21:15:44.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.614+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5c1a7610 con 0x7f6e5c1016d0 2024-04-25T21:15:44.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.616+0000 7f6e58ff9640 1 -- 172.21.15.37:0/1461305559 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6e44042030 con 0x7f6e5c1016d0 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 Processor -- start 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 -- start start 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13f1800 0x557bf13f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e58ff9640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 0x7f6e480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557bf134b970 con 0x557bf13e6620 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557bf12e84a0 con 0x557bf13f1800 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e64e79740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557bf1347930 con 0x557bf13e6f40 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60650/0 (socket says 172.21.15.37:60650) 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2079963574 learned_addr learned my addr 172.21.15.37:0/2079963574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e58ff9640 1 -- 172.21.15.37:0/1461305559 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6e440f1080 con 0x7f6e5c1016d0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 0x7f6e480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5affd640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13f1800 0x557bf13f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2079963574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 msgr2=0x557bf13f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2079963574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13f1800 msgr2=0x557bf13f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13f1800 0x557bf13f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2079963574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf105e210 con 0x557bf13e6620 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.617+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.618+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012cb0 tx=0x7f6e54010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712692c87244c3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.618+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 0x7f6e480a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009fd0 tx=0x7f6e5400c000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.618+0000 7f6e51ffb640 1 -- 172.21.15.37:0/2079963574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54023070 con 0x557bf13e6620 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.618+0000 7f6e51ffb640 1 -- 172.21.15.37:0/2079963574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e540124f0 con 0x557bf13e6620 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.618+0000 7f6e51ffb640 1 -- 172.21.15.37:0/2079963574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400ce20 con 0x557bf13e6620 2024-04-25T21:15:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- 172.21.15.37:0/2079963574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 msgr2=0x557bf13e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012cb0 tx=0x7f6e54010740 comp rx=0 tx=0).stop 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- 172.21.15.37:0/2079963574 shutdown_connections 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13f1800 0x557bf13f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2079963574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6620 0x557bf13e6a00 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- 172.21.15.37:0/2079963574 >> 172.21.15.37:0/2079963574 conn(0x557bf13e2b30 msgr2=0x557bf13e2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- 172.21.15.37:0/2079963574 shutdown_connections 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- 172.21.15.37:0/2079963574 wait complete. 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 -- start start 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 Processor -- start 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.619+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6620 0x557bf13f1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6f40 0x557bf13ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557bf13f43d0 con 0x557bf13f1800 2024-04-25T21:15:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557bf12e60b0 con 0x557bf13e6f40 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557bf13f4550 con 0x557bf13e6620 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 10 client.0 _ll_register_callbacks cb 0x557bf12d3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60654/0 (socket says 172.21.15.37:60654) 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 -- 172.21.15.37:0/1997481097 learned_addr learned my addr 172.21.15.37:0/1997481097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5affd640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6f40 0x557bf13ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6620 0x557bf13f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 -- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6620 msgr2=0x557bf13f1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6620 0x557bf13f1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 -- 172.21.15.37:0/1997481097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6f40 msgr2=0x557bf13ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6f40 0x557bf13ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 -- 172.21.15.37:0/1997481097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf112a620 con 0x557bf13f1800 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012440 tx=0x7f6e54012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd62ae157da7c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54023040 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6e54004a00 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.620+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e540124f0 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e54004c00 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 10 client.16227 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf1400380 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e540124f0 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e5400f040 con 0x557bf13f1800 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 10 client.16227 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 20 client.16227 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1401790 0x557bf1403b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 4 client.16227 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf1359390 con 0x557bf1401790 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.621+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1401790 0x557bf1403b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.622+0000 7f6e3affd640 10 client.16227 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:44.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.622+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1401790 0x557bf1403b80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2609425bbba2ab43 server_cookie=5ea5156355ab80f1 in_seq=0 out_seq=0 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e3affd640 1 -- 172.21.15.37:0/1997481097 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557bf1359390 con 0x557bf1401790 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e3affd640 10 client.16227 handle_command_reply: tid=0 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 client.16227 shutdown 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 msgr2=0x557bf13eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6e54012440 tx=0x7f6e54012980 comp rx=0 tx=0).stop 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 shutdown_connections 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1401790 0x557bf1403b80 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13eeb00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6f40 0x557bf13ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e3affd640 0 client.16227 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 --2- 172.21.15.37:0/1997481097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6620 0x557bf13f1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 >> 172.21.15.37:0/1997481097 conn(0x557bf13e2b30 msgr2=0x557bf13e8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 shutdown_connections 2024-04-25T21:15:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 1 -- 172.21.15.37:0/1997481097 wait complete. 2024-04-25T21:15:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.623+0000 7f6e64e79740 20 client.16227 trim_cache size 0 max 16384 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.638+0000 7f6e64e79740 1 Processor -- start 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.638+0000 7f6e64e79740 1 -- start start 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13f1800 0x557bf14096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1afd40 con 0x557bf13e6f40 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1b1880 con 0x557bf13e6620 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e64e79740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1b0410 con 0x557bf13f1800 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60670/0 (socket says 172.21.15.37:60670) 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 -- 172.21.15.37:0/3361917594 learned_addr learned my addr 172.21.15.37:0/3361917594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13f1800 0x557bf14096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 -- 172.21.15.37:0/3361917594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13f1800 msgr2=0x557bf14096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13f1800 0x557bf14096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 -- 172.21.15.37:0/3361917594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 msgr2=0x557bf13f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 -- 172.21.15.37:0/3361917594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf13eca60 con 0x557bf13e6f40 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e5affd640 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6e440ee880 tx=0x7f6e440045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e35e4e92fb9c3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e517fa640 1 -- 172.21.15.37:0/3361917594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e440d6070 con 0x557bf13e6f40 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e517fa640 1 -- 172.21.15.37:0/3361917594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e440bd720 con 0x557bf13e6f40 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.639+0000 7f6e517fa640 1 -- 172.21.15.37:0/3361917594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44049040 con 0x557bf13e6f40 2024-04-25T21:15:44.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- 172.21.15.37:0/3361917594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 msgr2=0x557bf1409180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6e440ee880 tx=0x7f6e440045a0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- 172.21.15.37:0/3361917594 shutdown_connections 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13f1800 0x557bf14096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13e6f40 0x557bf1409180 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 --2- 172.21.15.37:0/3361917594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- 172.21.15.37:0/3361917594 >> 172.21.15.37:0/3361917594 conn(0x557bf13e2b30 msgr2=0x557bf13ed540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- 172.21.15.37:0/3361917594 shutdown_connections 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- 172.21.15.37:0/3361917594 wait complete. 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.640+0000 7f6e64e79740 1 -- start start 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 Processor -- start 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1b11b0 con 0x557bf13f1800 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1b0ae0 con 0x557bf13e6620 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1add40 con 0x557bf13e6f40 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e64e79740 10 client.0 _ll_register_callbacks cb 0x7f6e540127c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60676/0 (socket says 172.21.15.37:60676) 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2783805090 learned_addr learned my addr 172.21.15.37:0/2783805090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5affd640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 msgr2=0x557bf13f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2783805090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 msgr2=0x557bf13f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 -- 172.21.15.37:0/2783805090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf13f7920 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e5bfff640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6e54010780 tx=0x7f6e54038730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94baecc3a532df1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.641+0000 7f6e3affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54023070 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f6e54004a00 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e440bd720 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6e540124f0 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 10 client.16233 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf1467810 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e440bd720 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6e54004e10 con 0x557bf13f1800 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 10 client.16233 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 20 client.16233 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1467f30 0x557bf146a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 4 client.16233 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:15:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.642+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bf1353200 con 0x557bf1467f30 2024-04-25T21:15:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.643+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1467f30 0x557bf146a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.643+0000 7f6e5b7fe640 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1467f30 0x557bf146a350 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3dad3a2d791106 server_cookie=76174e880ff0dee in_seq=0 out_seq=0 2024-04-25T21:15:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.643+0000 7f6e3affd640 10 client.16233 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e3affd640 1 -- 172.21.15.37:0/2783805090 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bf1353200 con 0x557bf1467f30 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e3affd640 10 client.16233 handle_command_reply: tid=0 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 client.16233 shutdown 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 msgr2=0x557bf13f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6e54010780 tx=0x7f6e54038730 comp rx=0 tx=0).stop 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 shutdown_connections 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557bf1467f30 0x557bf146a350 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.653+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bf13f1800 0x557bf13f72b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557bf13e6f40 0x557bf13f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e3affd640 0 client.16233 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 1 --2- 172.21.15.37:0/2783805090 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557bf13e6620 0x557bf13f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 >> 172.21.15.37:0/2783805090 conn(0x557bf13e2b30 msgr2=0x557bf13e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 shutdown_connections 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 1 -- 172.21.15.37:0/2783805090 wait complete. 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e64e79740 20 client.16233 trim_cache size 0 max 16384 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:15:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 msgr2=0x7f6e480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 0x7f6e480a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009fd0 tx=0x7f6e5400c000 comp rx=0 tx=0).stop 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 msgr2=0x7f6e5c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e44005940 tx=0x7f6e440042f0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 shutdown_connections 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6e480a3dc0 0x7f6e480a6280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e5c10e8d0 0x7f6e5c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e5c1016d0 0x7f6e5c1a0b60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 --2- 172.21.15.37:0/1461305559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e5c100d20 0x7f6e5c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 >> 172.21.15.37:0/1461305559 conn(0x7f6e5c077fd0 msgr2=0x7f6e5c10fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 shutdown_connections 2024-04-25T21:15:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:44.654+0000 7f6e6215d640 1 -- 172.21.15.37:0/1461305559 wait complete. 2024-04-25T21:15:45.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:45 smithi037 ceph-mon[29195]: pgmap v223: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T21:15:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:45 smithi177 ceph-mon[33043]: pgmap v223: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T21:15:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:45 smithi098 ceph-mon[32569]: pgmap v223: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.1 KiB/s wr, 14 op/s 2024-04-25T21:15:46.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:15:47.041 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:47.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:47 smithi037 ceph-mon[29195]: pgmap v224: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:47 smithi177 ceph-mon[33043]: pgmap v224: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:47.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:47 smithi098 ceph-mon[32569]: pgmap v224: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:48.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 -- 172.21.15.37:0/1369992647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd006bbb0 msgr2=0x7febd010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 --2- 172.21.15.37:0/1369992647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd006bbb0 0x7febd010f930 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7febc800b0a0 tx=0x7febc800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 -- 172.21.15.37:0/1369992647 shutdown_connections 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 --2- 172.21.15.37:0/1369992647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd006bbb0 0x7febd010f930 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 --2- 172.21.15.37:0/1369992647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd0108a60 0x7febd0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 --2- 172.21.15.37:0/1369992647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 0x7febd006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 -- 172.21.15.37:0/1369992647 >> 172.21.15.37:0/1369992647 conn(0x7febd00789e0 msgr2=0x7febd0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 -- 172.21.15.37:0/1369992647 shutdown_connections 2024-04-25T21:15:48.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.875+0000 7febd524d700 1 -- 172.21.15.37:0/1369992647 wait complete. 2024-04-25T21:15:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 Processor -- start 2024-04-25T21:15:48.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 -- start start 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 0x7febd019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd006bbb0 0x7febd019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febd0112ca0 con 0x7febd0108a60 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7febd0112b20 con 0x7febd006bbb0 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.876+0000 7febd524d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7febd0112e20 con 0x7febd006b1f0 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35112/0 (socket says 172.21.15.37:35112) 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 -- 172.21.15.37:0/1997268841 learned_addr learned my addr 172.21.15.37:0/1997268841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febc7fff700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd006bbb0 0x7febd019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febced9d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 0x7febd019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 -- 172.21.15.37:0/1997268841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 msgr2=0x7febd019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 0x7febd019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 -- 172.21.15.37:0/1997268841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd006bbb0 msgr2=0x7febd019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd006bbb0 0x7febd019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 -- 172.21.15.37:0/1997268841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd01a01e0 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febcf59e700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7febc800b830 tx=0x7febc8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8021070 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd01a0470 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc8007e30 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc80047d0 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.877+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a7d50 con 0x7febd0108a60 2024-04-25T21:15:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.878+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd01041a0 con 0x7febd0108a60 2024-04-25T21:15:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.880+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febc8007980 con 0x7febd0108a60 2024-04-25T21:15:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.880+0000 7febccd99700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 0x7febbc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.880+0000 7febced9d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 0x7febbc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:48.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.880+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7febc80d18f0 con 0x7febd0108a60 2024-04-25T21:15:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.881+0000 7febced9d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 0x7febbc0a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febc000dad0 tx=0x7febc000d4b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:48.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:48.884+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febc809dfc0 con 0x7febd0108a60 2024-04-25T21:15:49.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.138+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd01a7f20 con 0x7febd0108a60 2024-04-25T21:15:49.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.139+0000 7febccd99700 1 -- 172.21.15.37:0/1997268841 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7febc809d8c0 con 0x7febd0108a60 2024-04-25T21:15:49.142 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:15:49.142 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 msgr2=0x7febbc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 0x7febbc0a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febc000dad0 tx=0x7febc000d4b0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 msgr2=0x7febd019faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7febc800b830 tx=0x7febc8004ce0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 shutdown_connections 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7febbc0a3db0 0x7febbc0a6270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd0108a60 0x7febd019faa0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7febd006b1f0 0x7febd019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 --2- 172.21.15.37:0/1997268841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7febd006bbb0 0x7febd019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:49.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.141+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 >> 172.21.15.37:0/1997268841 conn(0x7febd00789e0 msgr2=0x7febd010dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:49.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.142+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 shutdown_connections 2024-04-25T21:15:49.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:49.142+0000 7febd524d700 1 -- 172.21.15.37:0/1997268841 wait complete. 2024-04-25T21:15:49.144 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:15:49.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:49 smithi037 ceph-mon[29195]: pgmap v225: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:49.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:49 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1997268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:49.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 8802b09a --release --await' 2024-04-25T21:15:49.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 8802b09a --release --await 2024-04-25T21:15:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:49 smithi177 ceph-mon[33043]: pgmap v225: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:49 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1997268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:49 smithi098 ceph-mon[32569]: pgmap v225: 97 pgs: 97 active+clean; 4.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:15:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:49 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1997268841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:50.078 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:51.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:51 smithi037 ceph-mon[29195]: pgmap v226: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T21:15:51.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:51 smithi177 ceph-mon[33043]: pgmap v226: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T21:15:51.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 -- 172.21.15.37:0/669907365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 msgr2=0x7faac406b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 --2- 172.21.15.37:0/669907365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac406b550 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faab4009990 tx=0x7faab4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 -- 172.21.15.37:0/669907365 shutdown_connections 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 --2- 172.21.15.37:0/669907365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 0x7faac4111910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 --2- 172.21.15.37:0/669907365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac406b550 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 --2- 172.21.15.37:0/669907365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 0x7faac410efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 -- 172.21.15.37:0/669907365 >> 172.21.15.37:0/669907365 conn(0x7faac40788f0 msgr2=0x7faac4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 -- 172.21.15.37:0/669907365 shutdown_connections 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.886+0000 7faac8d95700 1 -- 172.21.15.37:0/669907365 wait complete. 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 Processor -- start 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 -- start start 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 0x7faac419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 0x7faac419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faac4114e90 con 0x7faac406b170 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faac4114d10 con 0x7faac410f520 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac8d95700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac4115010 con 0x7faac406ba90 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35130/0 (socket says 172.21.15.37:35130) 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac259c700 1 -- 172.21.15.37:0/80682878 learned_addr learned my addr 172.21.15.37:0/80682878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac1d9b700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 0x7faac419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.887+0000 7faac2d9d700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 0x7faac419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 -- 172.21.15.37:0/80682878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 msgr2=0x7faac419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 0x7faac419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 -- 172.21.15.37:0/80682878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 msgr2=0x7faac419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 0x7faac419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 -- 172.21.15.37:0/80682878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac41a0390 con 0x7faac406b170 2024-04-25T21:15:51.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac259c700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faab37fe700 1 -- 172.21.15.37:0/80682878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4027070 con 0x7faac406b170 2024-04-25T21:15:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faab37fe700 1 -- 172.21.15.37:0/80682878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab4003f20 con 0x7faac406b170 2024-04-25T21:15:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faab37fe700 1 -- 172.21.15.37:0/80682878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4022cf0 con 0x7faac406b170 2024-04-25T21:15:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac41a0620 con 0x7faac406b170 2024-04-25T21:15:51.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.888+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac41a7f00 con 0x7faac406b170 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.891+0000 7faab37fe700 1 -- 172.21.15.37:0/80682878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faab402a070 con 0x7faac406b170 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.891+0000 7faab37fe700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 0x7faaac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.892+0000 7faab37fe700 1 -- 172.21.15.37:0/80682878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7faab40d1600 con 0x7faac406b170 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.892+0000 7faac1d9b700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 0x7faaac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.892+0000 7faac1d9b700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 0x7faaac0a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faac4101560 tx=0x7faab800b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.892+0000 7faad361eb80 1 Processor -- start 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.892+0000 7faad361eb80 1 -- start start 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e083ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 0x56143e0865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56143dfcbe30 con 0x56143e0789f0 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56143df72a60 con 0x56143e0841e0 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faad361eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56143dfc7df0 con 0x56143e079310 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35142/0 (socket says 172.21.15.37:35142) 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 -- 172.21.15.37:0/860255081 learned_addr learned my addr 172.21.15.37:0/860255081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:51.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac2d9d700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 0x56143e0865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac1d9b700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e083ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 -- 172.21.15.37:0/860255081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 msgr2=0x56143e083ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e083ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 -- 172.21.15.37:0/860255081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 msgr2=0x56143e0865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 0x56143e0865d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 -- 172.21.15.37:0/860255081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143df199d0 con 0x56143e0789f0 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac2d9d700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 0x56143e0865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:51.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faac259c700 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faab40a5c80 tx=0x7faab409dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67223e2455b41470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.893+0000 7faab0ff9700 1 -- 172.21.15.37:0/860255081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40a61b0 con 0x56143e0789f0 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faab0ff9700 1 -- 172.21.15.37:0/860255081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab4022470 con 0x56143e0789f0 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faab0ff9700 1 -- 172.21.15.37:0/860255081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40d1110 con 0x56143e0789f0 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 -- 172.21.15.37:0/860255081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 msgr2=0x56143e078dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faab40a5c80 tx=0x7faab409dcd0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 -- 172.21.15.37:0/860255081 shutdown_connections 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0841e0 0x56143e0865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0789f0 0x56143e078dd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 --2- 172.21.15.37:0/860255081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e083ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 -- 172.21.15.37:0/860255081 >> 172.21.15.37:0/860255081 conn(0x56143dfa6790 msgr2=0x56143e074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 -- 172.21.15.37:0/860255081 shutdown_connections 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.894+0000 7faad361eb80 1 -- 172.21.15.37:0/860255081 wait complete. 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 -- start start 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 Processor -- start 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e0839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e080ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56143dfcbe30 con 0x56143e0841e0 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56143df6acd0 con 0x56143e079310 2024-04-25T21:15:51.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56143e086b10 con 0x56143e0789f0 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faad361eb80 10 client.0 _ll_register_callbacks cb 0x56143df64a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faac259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e0839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35148/0 (socket says 172.21.15.37:35148) 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.895+0000 7faac2d9d700 1 -- 172.21.15.37:0/2823596497 learned_addr learned my addr 172.21.15.37:0/2823596497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac1d9b700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e080ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 -- 172.21.15.37:0/2823596497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 msgr2=0x56143e080ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e080ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 -- 172.21.15.37:0/2823596497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 msgr2=0x56143e081220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 -- 172.21.15.37:0/2823596497 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143df83370 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac2d9d700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faac259c700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e0839e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faab409d560 tx=0x7faab409da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd295a87574b577c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40ba070 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab4022470 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7faab40d1320 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faab40a61b0 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faad361eb80 10 client.25741 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.896+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56143e091170 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab4022470 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faab40040d0 con 0x56143e0789f0 2024-04-25T21:15:51.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 10 client.25741 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 10 client.25741 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 20 client.25741 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e093640 0x56143e095a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 4 client.25741 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143dfd9620 con 0x56143e093640 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.897+0000 7faac1d9b700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e093640 0x56143e095a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faac1d9b700 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e093640 0x56143e095a30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5584b0c7a2c52f server_cookie=744b54792a021582 in_seq=0 out_seq=0 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faaa9ffb700 10 client.25741 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faaa9ffb700 1 -- 172.21.15.37:0/2823596497 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56143dfd9620 con 0x56143e093640 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faaa9ffb700 10 client.25741 handle_command_reply: tid=0 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 client.25741 shutdown 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 msgr2=0x56143e0839e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e0839e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faab409d560 tx=0x7faab409da60 comp rx=0 tx=0).stop 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 shutdown_connections 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e093640 0x56143e095a30 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e081220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e0839e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faaa9ffb700 0 client.25741 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 --2- 172.21.15.37:0/2823596497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e080ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.898+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 >> 172.21.15.37:0/2823596497 conn(0x56143dfa6790 msgr2=0x56143e074b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.899+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 shutdown_connections 2024-04-25T21:15:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.899+0000 7faad361eb80 1 -- 172.21.15.37:0/2823596497 wait complete. 2024-04-25T21:15:51.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.899+0000 7faad361eb80 20 client.25741 trim_cache size 0 max 16384 2024-04-25T21:15:51.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:51 smithi098 ceph-mon[32569]: pgmap v226: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.3 KiB/s wr, 14 op/s 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 Processor -- start 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 -- start start 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0789f0 0x56143e082c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e09b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faac41af3a0 con 0x56143e0841e0 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b0ee0 con 0x56143e0789f0 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faad361eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac41afa70 con 0x56143e079310 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35154/0 (socket says 172.21.15.37:35154) 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faac2d9d700 1 -- 172.21.15.37:0/2761616010 learned_addr learned my addr 172.21.15.37:0/2761616010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faac259c700 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0789f0 0x56143e082c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.918+0000 7faac1d9b700 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e09b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 -- 172.21.15.37:0/2761616010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 msgr2=0x56143e09b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e09b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 -- 172.21.15.37:0/2761616010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0789f0 msgr2=0x56143e082c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0789f0 0x56143e082c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 -- 172.21.15.37:0/2761616010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143e0761b0 con 0x56143e0841e0 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faac2d9d700 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faabc00a750 tx=0x7faabc00a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42fe553b8355e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faaabfff700 1 -- 172.21.15.37:0/2761616010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc017630 con 0x56143e0841e0 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faaabfff700 1 -- 172.21.15.37:0/2761616010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faabc017c80 con 0x56143e0841e0 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faaabfff700 1 -- 172.21.15.37:0/2761616010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc016d60 con 0x56143e0841e0 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faad361eb80 1 -- 172.21.15.37:0/2761616010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 msgr2=0x56143e09b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.919+0000 7faad361eb80 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faabc00a750 tx=0x7faabc00a780 comp rx=0 tx=0).stop 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- 172.21.15.37:0/2761616010 shutdown_connections 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e09b6b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e0789f0 0x56143e082c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- 172.21.15.37:0/2761616010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e079310 0x56143e09b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- 172.21.15.37:0/2761616010 >> 172.21.15.37:0/2761616010 conn(0x56143dfa6790 msgr2=0x56143e082510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- 172.21.15.37:0/2761616010 shutdown_connections 2024-04-25T21:15:51.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- 172.21.15.37:0/2761616010 wait complete. 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- start start 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 Processor -- start 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e08c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e08a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b0810 con 0x56143e0841e0 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b0140 con 0x56143e079310 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7faac41ad360 con 0x56143e0789f0 2024-04-25T21:15:51.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faad361eb80 10 client.0 _ll_register_callbacks cb 0x56143df64a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faac259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e08c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.920+0000 7faac2d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35160/0 (socket says 172.21.15.37:35160) 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac259c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e08c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:48056/0 (socket says 172.21.15.37:48056) 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 -- 172.21.15.37:0/3084940989 learned_addr learned my addr 172.21.15.37:0/3084940989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac1d9b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e08a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 -- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 msgr2=0x56143e08c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e08c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 -- 172.21.15.37:0/3084940989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 msgr2=0x56143e08a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e08a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 -- 172.21.15.37:0/3084940989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143e0f9a00 con 0x56143e0841e0 2024-04-25T21:15:51.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac1d9b700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e08a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faac2d9d700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faabc016800 tx=0x7faabc016c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c085dc4229a2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faaa9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc026070 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7faabc017840 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faabc017c80 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faabc031050 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faad361eb80 10 client.16257 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.921+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56143e0fab40 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faabc017c80 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faabc01fc70 con 0x56143e0841e0 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 10 client.16257 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:51.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 10 client.16257 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 20 client.16257 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e0fb160 0x56143e0fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 4 client.16257 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8802b09a", "release": true, "await": true}] 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8802b09a", "release": true, "await": true}) -- 0x56143dfd36c0 con 0x56143e0fb160 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faac259c700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e0fb160 0x56143e0fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faaa9ffb700 10 client.16257 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:51.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.922+0000 7faac259c700 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e0fb160 0x56143e0fd580 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2276a974af36c4a server_cookie=fcc576c21cddb124 in_seq=0 out_seq=0 2024-04-25T21:15:51.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faaa9ffb700 1 -- 172.21.15.37:0/3084940989 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56143dfd36c0 con 0x56143e0fb160 2024-04-25T21:15:51.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faaa9ffb700 10 client.16257 handle_command_reply: tid=0 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 client.16257 shutdown 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 msgr2=0x56143e08a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faabc016800 tx=0x7faabc016c50 comp rx=0 tx=0).stop 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 shutdown_connections 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56143e0fb160 0x56143e0fd580 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56143e0841e0 0x56143e08a9f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56143e0789f0 0x56143e08c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 --2- 172.21.15.37:0/3084940989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56143e079310 0x56143e08a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faaa9ffb700 0 client.16257 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 >> 172.21.15.37:0/3084940989 conn(0x56143dfa6790 msgr2=0x56143e09a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 shutdown_connections 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 1 -- 172.21.15.37:0/3084940989 wait complete. 2024-04-25T21:15:51.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.926+0000 7faad361eb80 20 client.16257 trim_cache size 0 max 16384 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 20, 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "8802b09a": { 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 20, 2024-04-25T21:15:51.932 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.8, 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:51.933 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:51.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:51.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:51.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:51.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:51.934 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 msgr2=0x7faaac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 0x7faaac0a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faac4101560 tx=0x7faab800b3c0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 msgr2=0x7faac419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab40047b0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 shutdown_connections 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7faaac0a3db0 0x7faaac0a6270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac410f520 0x7faac419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faac406b170 0x7faac419b380 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 --2- 172.21.15.37:0/80682878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faac406ba90 0x7faac419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:51.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 >> 172.21.15.37:0/80682878 conn(0x7faac40788f0 msgr2=0x7faac4110b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:51.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 shutdown_connections 2024-04-25T21:15:51.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:51.927+0000 7faac8d95700 1 -- 172.21.15.37:0/80682878 wait complete. 2024-04-25T21:15:52.760 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8802b09a', seconds elapsed: 41.8 2024-04-25T21:15:52.760 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-04-25T21:15:52.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:15:53.014 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:53.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:53 smithi037 ceph-mon[29195]: pgmap v227: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:15:53.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:53 smithi177 ceph-mon[33043]: pgmap v227: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:15:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:53 smithi098 ceph-mon[32569]: pgmap v227: 97 pgs: 97 active+clean; 4.3 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 -- 172.21.15.37:0/288527824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 msgr2=0x7f15d0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 --2- 172.21.15.37:0/288527824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d0108e30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f15c0009990 tx=0x7f15c0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 -- 172.21.15.37:0/288527824 shutdown_connections 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 --2- 172.21.15.37:0/288527824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 0x7f15d010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 --2- 172.21.15.37:0/288527824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d0108e30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 --2- 172.21.15.37:0/288527824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 0x7f15d0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 -- 172.21.15.37:0/288527824 >> 172.21.15.37:0/288527824 conn(0x7f15d0079ba0 msgr2=0x7f15d0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 -- 172.21.15.37:0/288527824 shutdown_connections 2024-04-25T21:15:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.689+0000 7f15d63a8700 1 -- 172.21.15.37:0/288527824 wait complete. 2024-04-25T21:15:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 Processor -- start 2024-04-25T21:15:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 -- start start 2024-04-25T21:15:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 0x7f15d019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 0x7f15d019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0112c10 con 0x7f15d0108a50 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0112a90 con 0x7f15d0103450 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d63a8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0112d90 con 0x7f15d0102a90 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35164/0 (socket says 172.21.15.37:35164) 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 -- 172.21.15.37:0/271688009 learned_addr learned my addr 172.21.15.37:0/271688009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15cf7fe700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 0x7f15d019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15cffff700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 0x7f15d019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 -- 172.21.15.37:0/271688009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 msgr2=0x7f15d019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 0x7f15d019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 -- 172.21.15.37:0/271688009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 msgr2=0x7f15d019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.690+0000 7f15d4945700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 0x7f15d019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15d4945700 1 -- 172.21.15.37:0/271688009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d01a0330 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15d4945700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007ee0 tx=0x7f15c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c800dbd0 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c800dd70 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d01a0620 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8005990 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.691+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d01a7ea0 con 0x7f15d0108a50 2024-04-25T21:15:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.692+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d0069730 con 0x7f15d0108a50 2024-04-25T21:15:54.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.694+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15c801a020 con 0x7f15d0108a50 2024-04-25T21:15:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.694+0000 7f15cd7fa700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.695+0000 7f15cffff700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.695+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f15c80d05a0 con 0x7f15d0108a50 2024-04-25T21:15:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.695+0000 7f15cffff700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 0x7f15b80a62f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006010 tx=0x7f15c001a040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:54.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.698+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15c8010070 con 0x7f15d0108a50 2024-04-25T21:15:54.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.957+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d01a8130 con 0x7f15d0108a50 2024-04-25T21:15:54.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.958+0000 7f15cd7fa700 1 -- 172.21.15.37:0/271688009 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f15c809cbe0 con 0x7f15d0108a50 2024-04-25T21:15:54.958 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:15:54.959 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:15:54.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 msgr2=0x7f15b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 0x7f15b80a62f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006010 tx=0x7f15c001a040 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 msgr2=0x7f15d019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007ee0 tx=0x7f15c800d3b0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 shutdown_connections 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f15b80a3e30 0x7f15b80a62f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15d0108a50 0x7f15d019fbf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f15d0102a90 0x7f15d019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 --2- 172.21.15.37:0/271688009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f15d0103450 0x7f15d019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 >> 172.21.15.37:0/271688009 conn(0x7f15d0079ba0 msgr2=0x7f15d010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 shutdown_connections 2024-04-25T21:15:54.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:54.960+0000 7f15d63a8700 1 -- 172.21.15.37:0/271688009 wait complete. 2024-04-25T21:15:54.962 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:15:55.626 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0' 2024-04-25T21:15:55.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 89.8 --await-for 42.0 2024-04-25T21:15:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:55 smithi177 ceph-mon[33043]: pgmap v228: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T21:15:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:55 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/271688009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:55.877 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:15:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:55 smithi098 ceph-mon[32569]: pgmap v228: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T21:15:55.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:55 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/271688009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:55 smithi037 ceph-mon[29195]: pgmap v228: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-25T21:15:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:55 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/271688009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:15:57.567 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:57 smithi037 ceph-mon[29195]: pgmap v229: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 -- 172.21.15.37:0/3624746187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 msgr2=0x7f024c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 --2- 172.21.15.37:0/3624746187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c10e780 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f024400b0a0 tx=0x7f024400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 -- 172.21.15.37:0/3624746187 shutdown_connections 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 --2- 172.21.15.37:0/3624746187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c10e780 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 --2- 172.21.15.37:0/3624746187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 0x7f024c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 --2- 172.21.15.37:0/3624746187 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 -- 172.21.15.37:0/3624746187 >> 172.21.15.37:0/3624746187 conn(0x7f024c0788f0 msgr2=0x7f024c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 -- 172.21.15.37:0/3624746187 shutdown_connections 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.687+0000 7f02544e4700 1 -- 172.21.15.37:0/3624746187 wait complete. 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 Processor -- start 2024-04-25T21:15:57.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 -- start start 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 0x7f024c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f024c111b80 con 0x7f024c1026e0 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f024c111a00 con 0x7f024c101d20 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f02544e4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f024c111d00 con 0x7f024c1079d0 2024-04-25T21:15:57.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f0251a7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f0251a7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35174/0 (socket says 172.21.15.37:35174) 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f0251a7f700 1 -- 172.21.15.37:0/4181794786 learned_addr learned my addr 172.21.15.37:0/4181794786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f0252280700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.688+0000 7f0252a81700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 0x7f024c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 -- 172.21.15.37:0/4181794786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 msgr2=0x7f024c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 0x7f024c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 -- 172.21.15.37:0/4181794786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 msgr2=0x7f024c199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 -- 172.21.15.37:0/4181794786 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024c1a53c0 con 0x7f024c1026e0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0252280700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f0251a7f700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f023c00e9a0 tx=0x7f023c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f02437fe700 1 -- 172.21.15.37:0/4181794786 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c00ca10 con 0x7f024c1026e0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f02437fe700 1 -- 172.21.15.37:0/4181794786 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f023c00cbb0 con 0x7f024c1026e0 2024-04-25T21:15:57.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f02437fe700 1 -- 172.21.15.37:0/4181794786 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c018680 con 0x7f024c1026e0 2024-04-25T21:15:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024c1a55f0 con 0x7f024c1026e0 2024-04-25T21:15:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.689+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f024c1a59f0 con 0x7f024c1026e0 2024-04-25T21:15:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.691+0000 7f02437fe700 1 -- 172.21.15.37:0/4181794786 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f023c010430 con 0x7f024c1026e0 2024-04-25T21:15:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.692+0000 7f02437fe700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 0x7f02380a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.692+0000 7f0252280700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 0x7f02380a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.693+0000 7f02437fe700 1 -- 172.21.15.37:0/4181794786 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f023c0d0960 con 0x7f024c1026e0 2024-04-25T21:15:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.693+0000 7f0252280700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 0x7f02380a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0248009fd0 tx=0x7f0248009260 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.693+0000 7f025ed6db80 1 Processor -- start 2024-04-25T21:15:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 -- start start 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c933cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c93479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5642c928fe20 con 0x5642c9347f00 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642c9236ac0 con 0x5642c933d000 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f025ed6db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5642c928bde0 con 0x5642c933c6e0 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f0252a81700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f0252a81700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35182/0 (socket says 172.21.15.37:35182) 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f0252a81700 1 -- 172.21.15.37:0/1910543268 learned_addr learned my addr 172.21.15.37:0/1910543268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f0252280700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c933cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.694+0000 7f0251a7f700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c93479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 -- 172.21.15.37:0/1910543268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 msgr2=0x5642c933cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c933cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 -- 172.21.15.37:0/1910543268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 msgr2=0x5642c93479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c93479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 -- 172.21.15.37:0/1910543268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c91fa8d0 con 0x5642c9347f00 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252280700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c933cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0252a81700 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0244009df0 tx=0x7f0244009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0621086cad0f034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0240d70700 1 -- 172.21.15.37:0/1910543268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024400bab0 con 0x5642c9347f00 2024-04-25T21:15:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0240d70700 1 -- 172.21.15.37:0/1910543268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024400bc50 con 0x5642c9347f00 2024-04-25T21:15:57.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.695+0000 7f0240d70700 1 -- 172.21.15.37:0/1910543268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244007e20 con 0x5642c9347f00 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 -- 172.21.15.37:0/1910543268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 msgr2=0x5642c934a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0244009df0 tx=0x7f0244009e20 comp rx=0 tx=0).stop 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 -- 172.21.15.37:0/1910543268 shutdown_connections 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c934a2f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c933cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 --2- 172.21.15.37:0/1910543268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c93479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 -- 172.21.15.37:0/1910543268 >> 172.21.15.37:0/1910543268 conn(0x5642c9269a20 msgr2=0x5642c9338ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 -- 172.21.15.37:0/1910543268 shutdown_connections 2024-04-25T21:15:57.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.696+0000 7f025ed6db80 1 -- 172.21.15.37:0/1910543268 wait complete. 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 -- start start 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 Processor -- start 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9347640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 0x5642c9344e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5642c934a930 con 0x5642c933d000 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5642c9249740 con 0x5642c9347f00 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5642c934aab0 con 0x5642c933c6e0 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f025ed6db80 10 client.0 _ll_register_callbacks cb 0x5642c9228ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.697+0000 7f0251a7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35184/0 (socket says 172.21.15.37:35184) 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 -- 172.21.15.37:0/2830116634 learned_addr learned my addr 172.21.15.37:0/2830116634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0252280700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9347640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0252a81700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 0x5642c9344e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 -- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 msgr2=0x5642c9347640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9347640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 -- 172.21.15.37:0/2830116634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 msgr2=0x5642c9344e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 0x5642c9344e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 -- 172.21.15.37:0/2830116634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c91dda60 con 0x5642c933d000 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0252a81700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 0x5642c9344e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0251a7f700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f023c098d00 tx=0x7f023c09d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c6f4e244d3be86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0235ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c09cd00 con 0x5642c933d000 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f023c09dbd0 con 0x5642c933d000 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024400bc50 con 0x5642c933d000 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f023c0ba020 con 0x5642c933d000 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f025ed6db80 10 client.16275 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:57.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.698+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642c9355200 con 0x5642c933d000 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024400bc50 con 0x5642c933d000 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f023c010720 con 0x5642c933d000 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 10 client.16275 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 10 client.16275 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 20 client.16275 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c935a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 4 client.16275 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642c929d610 con 0x5642c9357c20 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.699+0000 7f0252280700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c935a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.700+0000 7f0252280700 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c935a040 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66285de42fc596 server_cookie=9726e04096003abb in_seq=0 out_seq=0 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.700+0000 7f0235ffb700 10 client.16275 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f0235ffb700 1 -- 172.21.15.37:0/2830116634 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642c929d610 con 0x5642c9357c20 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f0235ffb700 10 client.16275 handle_command_reply: tid=0 2024-04-25T21:15:57.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 client.16275 shutdown 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 msgr2=0x5642c9344940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f023c098d00 tx=0x7f023c09d020 comp rx=0 tx=0).stop 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 shutdown_connections 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c935a040 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c9347f00 0x5642c9344e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f0235ffb700 0 client.16275 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9347640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 --2- 172.21.15.37:0/2830116634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933d000 0x5642c9344940 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 >> 172.21.15.37:0/2830116634 conn(0x5642c9269a20 msgr2=0x5642c933f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 shutdown_connections 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 1 -- 172.21.15.37:0/2830116634 wait complete. 2024-04-25T21:15:57.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.701+0000 7f025ed6db80 20 client.16275 trim_cache size 0 max 16384 2024-04-25T21:15:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.721+0000 7f025ed6db80 1 Processor -- start 2024-04-25T21:15:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.721+0000 7f025ed6db80 1 -- start start 2024-04-25T21:15:57.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.721+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9337b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.721+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c935f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f025ed6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1ad160 con 0x5642c9347f00 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f025ed6db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1aeca0 con 0x5642c933d000 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f025ed6db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1ad830 con 0x5642c933c6e0 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35190/0 (socket says 172.21.15.37:35190) 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 -- 172.21.15.37:0/3562457757 learned_addr learned my addr 172.21.15.37:0/3562457757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252280700 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9337b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0251a7f700 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c935f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 -- 172.21.15.37:0/3562457757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 msgr2=0x5642c9337b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9337b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 -- 172.21.15.37:0/3562457757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 msgr2=0x5642c935f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c935f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 -- 172.21.15.37:0/3562457757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c935a3b0 con 0x5642c9347f00 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0252a81700 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0244009a20 tx=0x7f024400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4044f37bdc87dcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.722+0000 7f0237fff700 1 -- 172.21.15.37:0/3562457757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244003a00 con 0x5642c9347f00 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f0237fff700 1 -- 172.21.15.37:0/3562457757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244003ba0 con 0x5642c9347f00 2024-04-25T21:15:57.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f0237fff700 1 -- 172.21.15.37:0/3562457757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024402f470 con 0x5642c9347f00 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 -- 172.21.15.37:0/3562457757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 msgr2=0x5642c935fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0244009a20 tx=0x7f024400b040 comp rx=0 tx=0).stop 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 -- 172.21.15.37:0/3562457757 shutdown_connections 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c9347f00 0x5642c935fe90 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c933c6e0 0x5642c9337b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 --2- 172.21.15.37:0/3562457757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c935f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 -- 172.21.15.37:0/3562457757 >> 172.21.15.37:0/3562457757 conn(0x5642c9269a20 msgr2=0x5642c9337440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.723+0000 7f025ed6db80 1 -- 172.21.15.37:0/3562457757 shutdown_connections 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 -- 172.21.15.37:0/3562457757 wait complete. 2024-04-25T21:15:57.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 -- start start 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 Processor -- start 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c934c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 0x5642c93bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1ae5d0 con 0x5642c933c6e0 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1adf00 con 0x5642c933d000 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f024c1ab120 con 0x5642c9347f00 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f0252280700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f025ed6db80 10 client.0 _ll_register_callbacks cb 0x5642c9228ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f0252a81700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 0x5642c93bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f0252280700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35198/0 (socket says 172.21.15.37:35198) 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f0252280700 1 -- 172.21.15.37:0/562123389 learned_addr learned my addr 172.21.15.37:0/562123389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.724+0000 7f0252a81700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 0x5642c93bd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:48124/0 (socket says 172.21.15.37:48124) 2024-04-25T21:15:57.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0251a7f700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c934c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 -- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 msgr2=0x5642c93bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 0x5642c93bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 -- 172.21.15.37:0/562123389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 msgr2=0x5642c934c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c934c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 -- 172.21.15.37:0/562123389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c93bd700 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0252280700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x5642c935a8c0 tx=0x7f02480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a537c905515239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0235ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024802a380 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7f024802a520 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244003ba0 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f024802a720 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f025ed6db80 10 client.16281 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.725+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642c93be9b0 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244003ba0 con 0x5642c933c6e0 2024-04-25T21:15:57.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f024803b480 con 0x5642c933c6e0 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 10 client.16281 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 10 client.16281 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 20 client.16281 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c93c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 4 client.16281 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}] 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.8, "await_for": 42.0}) -- 0x5642c92976b0 con 0x5642c9357c20 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.726+0000 7f0251a7f700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c93c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.727+0000 7f0235ffb700 10 client.16281 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.727+0000 7f0251a7f700 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c93c10f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2e18accff5939 server_cookie=c173d782b2c76a25 in_seq=0 out_seq=0 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.741+0000 7f0235ffb700 1 -- 172.21.15.37:0/562123389 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5642c92976b0 con 0x5642c9357c20 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.741+0000 7f0235ffb700 10 client.16281 handle_command_reply: tid=0 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.741+0000 7f025ed6db80 1 client.16281 shutdown 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.741+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 msgr2=0x5642c934bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.741+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x5642c935a8c0 tx=0x7f02480047c0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 shutdown_connections 2024-04-25T21:15:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5642c9357c20 0x5642c93c10f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5642c9347f00 0x5642c93bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5642c933c6e0 0x5642c934bdd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 --2- 172.21.15.37:0/562123389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5642c933d000 0x5642c934c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 >> 172.21.15.37:0/562123389 conn(0x5642c9269a20 msgr2=0x5642c9337cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f0235ffb700 0 client.16281 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 shutdown_connections 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 1 -- 172.21.15.37:0/562123389 wait complete. 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f025ed6db80 20 client.16281 trim_cache size 0 max 16384 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:15:57.743 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 22, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "10a4283f": { 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 22, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.8, 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:15:57.744 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:15:57.745 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 msgr2=0x7f02380a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 0x7f02380a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0248009fd0 tx=0x7f0248009260 comp rx=0 tx=0).stop 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 msgr2=0x7f024c19a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.742+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f023c00e9a0 tx=0x7f023c00ecb0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 shutdown_connections 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f02380a3ed0 0x7f02380a6390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f024c1079d0 0x7f024c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f024c101d20 0x7f024c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 --2- 172.21.15.37:0/4181794786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f024c1026e0 0x7f024c19a340 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 >> 172.21.15.37:0/4181794786 conn(0x7f024c0788f0 msgr2=0x7f024c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 shutdown_connections 2024-04-25T21:15:57.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:15:57.743+0000 7f02544e4700 1 -- 172.21.15.37:0/4181794786 wait complete. 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.754+0000 7fc5b9d08640 1 -- 172.21.15.37:0/95541173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b41010f0 msgr2=0x7fc5b41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.754+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/95541173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b41010f0 0x7fc5b41014d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009900 tx=0x7fc5a402a410 comp rx=0 tx=0).stop 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 -- 172.21.15.37:0/95541173 shutdown_connections 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/95541173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b4106170 0x7fc5b410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/95541173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4101aa0 0x7fc5b4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/95541173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b41010f0 0x7fc5b41014d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 -- 172.21.15.37:0/95541173 >> 172.21.15.37:0/95541173 conn(0x7fc5b4077fd0 msgr2=0x7fc5b40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 -- 172.21.15.37:0/95541173 shutdown_connections 2024-04-25T21:15:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 -- 172.21.15.37:0/95541173 wait complete. 2024-04-25T21:15:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 Processor -- start 2024-04-25T21:15:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.755+0000 7fc5b9d08640 1 -- start start 2024-04-25T21:15:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b41010f0 0x7fc5b41975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4106170 0x7fc5b4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41110e0 con 0x7fc5b419bea0 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4110f60 con 0x7fc5b4106170 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4111260 con 0x7fc5b41010f0 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35214/0 (socket says 172.21.15.37:35214) 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 -- 172.21.15.37:0/2639433284 learned_addr learned my addr 172.21.15.37:0/2639433284 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b41010f0 0x7fc5b41975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4106170 0x7fc5b4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 -- 172.21.15.37:0/2639433284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b41010f0 msgr2=0x7fc5b41975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b41010f0 0x7fc5b41975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 -- 172.21.15.37:0/2639433284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4106170 msgr2=0x7fc5b4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4106170 0x7fc5b4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 -- 172.21.15.37:0/2639433284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b419c990 con 0x7fc5b419bea0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b3fff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00eaa0 tx=0x7fc5ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b17fa640 1 -- 172.21.15.37:0/2639433284 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00cdb0 con 0x7fc5b419bea0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b419cc80 con 0x7fc5b419bea0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.756+0000 7fc5b17fa640 1 -- 172.21.15.37:0/2639433284 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5ac005020 con 0x7fc5b419bea0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.757+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b41a44c0 con 0x7fc5b419bea0 2024-04-25T21:15:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.757+0000 7fc5b17fa640 1 -- 172.21.15.37:0/2639433284 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac0116d0 con 0x7fc5b419bea0 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.759+0000 7fc5bca1d740 1 Processor -- start 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.759+0000 7fc5b17fa640 1 -- 172.21.15.37:0/2639433284 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5ac0054a0 con 0x7fc5b419bea0 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.759+0000 7fc5bca1d740 1 -- start start 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558442810970 con 0x5584428b8a10 2024-04-25T21:15:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5584427ad4a0 con 0x5584428ad830 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55844280c930 con 0x5584428ae150 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35228/0 (socket says 172.21.15.37:35228) 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 -- 172.21.15.37:0/2783039395 learned_addr learned my addr 172.21.15.37:0/2783039395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5b17fa640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 0x7fc5a00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 -- 172.21.15.37:0/2783039395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 msgr2=0x5584428b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 -- 172.21.15.37:0/2783039395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 msgr2=0x5584428adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5abfff640 1 -- 172.21.15.37:0/2783039395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558442523210 con 0x5584428b8a10 2024-04-25T21:15:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.760+0000 7fc5b3fff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 0x7fc5a00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.761+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c00b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a604c6206edefb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.761+0000 7fc5b17fa640 1 -- 172.21.15.37:0/2639433284 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fc5ac0d0910 con 0x7fc5b419bea0 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.761+0000 7fc5a9ffb640 1 -- 172.21.15.37:0/2783039395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00e950 con 0x5584428b8a10 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.761+0000 7fc5a9ffb640 1 -- 172.21.15.37:0/2783039395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c010430 con 0x5584428b8a10 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.761+0000 7fc5a9ffb640 1 -- 172.21.15.37:0/2783039395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00f620 con 0x5584428b8a10 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5b3fff640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 0x7fc5a00a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0ce660 tx=0x7fc5ac06f3a0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2783039395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 msgr2=0x5584428bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005fd0 tx=0x7fc59c00b440 comp rx=0 tx=0).stop 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2783039395 shutdown_connections 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428b8a10 0x5584428bae00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2783039395 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2783039395 >> 172.21.15.37:0/2783039395 conn(0x5584428a9d40 msgr2=0x5584428aa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2783039395 shutdown_connections 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2783039395 wait complete. 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.762+0000 7fc5bca1d740 1 -- start start 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 Processor -- start 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ad830 0x5584428b8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584428bb640 con 0x5584428ae150 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5584427ab0b0 con 0x5584428b8a10 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5584428bb7c0 con 0x5584428ad830 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5bca1d740 10 client.0 _ll_register_callbacks cb 0x558442798d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35242/0 (socket says 172.21.15.37:35242) 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 -- 172.21.15.37:0/2543462730 learned_addr learned my addr 172.21.15.37:0/2543462730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ad830 0x5584428b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 -- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ad830 msgr2=0x5584428b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ad830 0x5584428b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 -- 172.21.15.37:0/2543462730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 msgr2=0x5584428b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5b37fe640 1 -- 172.21.15.37:0/2543462730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584425ef620 con 0x5584428ae150 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.763+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402a930 tx=0x7fc5a40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394bdfee63868483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4035040 con 0x5584428ae150 2024-04-25T21:15:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fc5a4004040 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c010430 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc5a4032e10 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 10 client.16293 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5584428c7640 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c010430 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5a4004850 con 0x5584428ae150 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 10 client.16293 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 20 client.16293 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5584428c8b00 0x5584428caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 4 client.16293 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.764+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55844281e390 con 0x5584428c8b00 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.765+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5584428c8b00 0x5584428caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.765+0000 7fc5abfff640 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5584428c8b00 0x5584428caf10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2ca5c7f3c11797 server_cookie=8409b657a31fcec6 in_seq=0 out_seq=0 2024-04-25T21:15:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.765+0000 7fc592ffd640 10 client.16293 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc592ffd640 1 -- 172.21.15.37:0/2543462730 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55844281e390 con 0x5584428c8b00 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc592ffd640 10 client.16293 handle_command_reply: tid=0 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 client.16293 shutdown 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 msgr2=0x5584428b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc5a402a930 tx=0x7fc5a40316d0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 shutdown_connections 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5584428c8b00 0x5584428caf10 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428b5700 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc592ffd640 0 client.16293 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/2543462730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ad830 0x5584428b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 >> 172.21.15.37:0/2543462730 conn(0x5584428a9d40 msgr2=0x5584428a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 shutdown_connections 2024-04-25T21:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 1 -- 172.21.15.37:0/2543462730 wait complete. 2024-04-25T21:15:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.766+0000 7fc5bca1d740 20 client.16293 trim_cache size 0 max 16384 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 Processor -- start 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 -- start start 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 0x5584428d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41ab920 con 0x5584428ae150 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41ad460 con 0x5584428ad830 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41abff0 con 0x5584428b8a10 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35258/0 (socket says 172.21.15.37:35258) 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5b37fe640 1 -- 172.21.15.37:0/555401488 learned_addr learned my addr 172.21.15.37:0/555401488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.782+0000 7fc5abfff640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 0x5584428d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5abfff640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 -- 172.21.15.37:0/555401488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 msgr2=0x5584428d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 0x5584428d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 -- 172.21.15.37:0/555401488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 msgr2=0x5584428c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 -- 172.21.15.37:0/555401488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584428b46b0 con 0x5584428ae150 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5abfff640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 0x5584428d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4006010 tx=0x7fc5a402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de21908ee2b06aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5a97fa640 1 -- 172.21.15.37:0/555401488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4035070 con 0x5584428ae150 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5a97fa640 1 -- 172.21.15.37:0/555401488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a4004950 con 0x5584428ae150 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.783+0000 7fc5a97fa640 1 -- 172.21.15.37:0/555401488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4052070 con 0x5584428ae150 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- 172.21.15.37:0/555401488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 msgr2=0x5584428a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4006010 tx=0x7fc5a402a410 comp rx=0 tx=0).stop 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- 172.21.15.37:0/555401488 shutdown_connections 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428b8a10 0x5584428d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ae150 0x5584428a6cb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/555401488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428ad830 0x5584428c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- 172.21.15.37:0/555401488 >> 172.21.15.37:0/555401488 conn(0x5584428a9d40 msgr2=0x5584428aa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- 172.21.15.37:0/555401488 shutdown_connections 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- 172.21.15.37:0/555401488 wait complete. 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 -- start start 2024-04-25T21:15:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.784+0000 7fc5bca1d740 1 Processor -- start 2024-04-25T21:15:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41acd90 con 0x5584428ad830 2024-04-25T21:15:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41ac6c0 con 0x5584428b8a10 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41a9920 con 0x5584428ae150 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 10 client.0 _ll_register_callbacks cb 0x7fc5a4032cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35272/0 (socket says 172.21.15.37:35272) 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 -- 172.21.15.37:0/3290450820 learned_addr learned my addr 172.21.15.37:0/3290450820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 -- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 msgr2=0x5584428be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 -- 172.21.15.37:0/3290450820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 msgr2=0x5584428bef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428bef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 -- 172.21.15.37:0/3290450820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584428adf80 con 0x5584428ad830 2024-04-25T21:15:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5abfff640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5584428aaed0 tx=0x7fc59c02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e2bf36c765c78b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c010da0 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8513+0+0 (secure 0 0 0) 0x7fc59c00bbf0 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.785+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a4004950 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc59c0050b0 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 10 client.16299 fetch_fsmap learned FSMap version 27 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55844292f1f0 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a4004950 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc59c0292d0 con 0x5584428ad830 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 10 client.16299 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 20 client.16299 populate_metadata read hostname 'smithi037' 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55844292f6c0 0x558442931ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:15:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 4 client.16299 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:15:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558442818200 con 0x55844292f6c0 2024-04-25T21:15:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.786+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55844292f6c0 0x558442931ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:15:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.787+0000 7fc5b37fe640 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55844292f6c0 0x558442931ab0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93ae9cff0b07570 server_cookie=d192781107cacf5f in_seq=0 out_seq=0 2024-04-25T21:15:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.787+0000 7fc592ffd640 10 client.16299 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc592ffd640 1 -- 172.21.15.37:0/3290450820 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558442818200 con 0x55844292f6c0 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc592ffd640 10 client.16299 handle_command_reply: tid=0 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 client.16299 shutdown 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 msgr2=0x5584428be480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x5584428aaed0 tx=0x7fc59c02b040 comp rx=0 tx=0).stop 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 shutdown_connections 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55844292f6c0 0x558442931ab0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5584428b8a10 0x5584428bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5584428ae150 0x5584428be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc592ffd640 0 client.16299 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 --2- 172.21.15.37:0/3290450820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584428ad830 0x5584428be480 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 >> 172.21.15.37:0/3290450820 conn(0x5584428a9d40 msgr2=0x5584428a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 shutdown_connections 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.798+0000 7fc5bca1d740 1 -- 172.21.15.37:0/3290450820 wait complete. 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5bca1d740 20 client.16299 trim_cache size 0 max 16384 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:15:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 msgr2=0x7fc5a00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 0x7fc5a00a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0ce660 tx=0x7fc5ac06f3a0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 msgr2=0x7fc5b419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00eaa0 tx=0x7fc5ac00ef60 comp rx=0 tx=0).stop 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 shutdown_connections 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5b419bea0 0x7fc5b419c350 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5b4106170 0x7fc5b4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5a00a3dc0 0x7fc5a00a6280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 --2- 172.21.15.37:0/2639433284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5b41010f0 0x7fc5b41975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:15:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 >> 172.21.15.37:0/2639433284 conn(0x7fc5b4077fd0 msgr2=0x7fc5b40fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:15:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 shutdown_connections 2024-04-25T21:15:57.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:15:57.799+0000 7fc5b9d08640 1 -- 172.21.15.37:0/2639433284 wait complete. 2024-04-25T21:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:57 smithi177 ceph-mon[33043]: pgmap v229: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:15:57.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:57 smithi098 ceph-mon[32569]: pgmap v229: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:15:58.510 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '10a4283f', quiesce took 5.7 seconds. Will release after: 29.8 2024-04-25T21:15:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:15:59 smithi177 ceph-mon[33043]: pgmap v230: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:15:59.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:15:59 smithi098 ceph-mon[32569]: pgmap v230: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:15:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:15:59 smithi037 ceph-mon[29195]: pgmap v230: 97 pgs: 97 active+clean; 4.4 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-25T21:16:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:01 smithi177 ceph-mon[33043]: pgmap v231: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-25T21:16:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:01 smithi098 ceph-mon[32569]: pgmap v231: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-25T21:16:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:01 smithi037 ceph-mon[29195]: pgmap v231: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-25T21:16:03.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:03 smithi037 ceph-mon[29195]: pgmap v232: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T21:16:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:03 smithi177 ceph-mon[33043]: pgmap v232: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T21:16:03.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:03 smithi098 ceph-mon[32569]: pgmap v232: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T21:16:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:05 smithi177 ceph-mon[33043]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:16:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:05 smithi098 ceph-mon[32569]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:16:05.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:05 smithi037 ceph-mon[29195]: pgmap v233: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:16:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:07 smithi177 ceph-mon[33043]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-04-25T21:16:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:07 smithi177 ceph-mon[33043]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:16:07.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:07 smithi098 ceph-mon[32569]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-04-25T21:16:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:07 smithi098 ceph-mon[32569]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:16:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:07 smithi037 ceph-mon[29195]: pgmap v234: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 60 KiB/s wr, 29 op/s 2024-04-25T21:16:07.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:07 smithi037 ceph-mon[29195]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T21:16:09.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:09 smithi177 ceph-mon[33043]: pgmap v236: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-04-25T21:16:09.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:09 smithi098 ceph-mon[32569]: pgmap v236: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-04-25T21:16:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:09 smithi037 ceph-mon[29195]: pgmap v236: 97 pgs: 97 active+clean; 4.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-04-25T21:16:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 -- 172.21.15.37:0/2021067876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 msgr2=0x7f001c110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2021067876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c110d00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0014009ca0 tx=0x7f001402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 -- 172.21.15.37:0/2021067876 shutdown_connections 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2021067876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c110d00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2021067876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c101680 0x7f001c10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2021067876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c100d60 0x7f001c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 -- 172.21.15.37:0/2021067876 >> 172.21.15.37:0/2021067876 conn(0x7f001c077fd0 msgr2=0x7f001c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 -- 172.21.15.37:0/2021067876 shutdown_connections 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 -- 172.21.15.37:0/2021067876 wait complete. 2024-04-25T21:16:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.899+0000 7f0021e8a640 1 Processor -- start 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 -- start start 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c100d60 0x7f001c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 0x7f001c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1143b0 con 0x7f001c10e910 2024-04-25T21:16:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f001c114230 con 0x7f001c101680 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f0021e8a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f001c114530 con 0x7f001c100d60 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39578/0 (socket says 172.21.15.37:39578) 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 -- 172.21.15.37:0/2339424730 learned_addr learned my addr 172.21.15.37:0/2339424730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001b7fe640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c100d60 0x7f001c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001affd640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 0x7f001c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 -- 172.21.15.37:0/2339424730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c100d60 msgr2=0x7f001c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c100d60 0x7f001c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 -- 172.21.15.37:0/2339424730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 msgr2=0x7f001c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 0x7f001c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001bfff640 1 -- 172.21.15.37:0/2339424730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c19b030 con 0x7f001c10e910 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.900+0000 7f001affd640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 0x7f001c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f001bfff640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f00140319f0 tx=0x7f0014031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f0018ff9640 1 -- 172.21.15.37:0/2339424730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014003bf0 con 0x7f001c10e910 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f0018ff9640 1 -- 172.21.15.37:0/2339424730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0014003d90 con 0x7f001c10e910 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c19b2c0 con 0x7f001c10e910 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f0018ff9640 1 -- 172.21.15.37:0/2339424730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001402fbc0 con 0x7f001c10e910 2024-04-25T21:16:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.901+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c1a7630 con 0x7f001c10e910 2024-04-25T21:16:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.903+0000 7f0018ff9640 1 -- 172.21.15.37:0/2339424730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f001402fd60 con 0x7f001c10e910 2024-04-25T21:16:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.903+0000 7f0024ba5740 1 Processor -- start 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 -- start start 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0018ff9640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 0x7f00040a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2787a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 0x5637d2794bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5637d26ec970 con 0x5637d2787f40 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5637d26894a0 con 0x5637d2792800 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0024ba5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5637d26e8930 con 0x5637d2787620 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 0x7f00040a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f0018ff9640 1 -- 172.21.15.37:0/2339424730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f00140f1c80 con 0x7f001c10e910 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39590/0 (socket says 172.21.15.37:39590) 2024-04-25T21:16:10.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 -- 172.21.15.37:0/3694337733 learned_addr learned my addr 172.21.15.37:0/3694337733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001bfff640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 0x5637d2794bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001b7fe640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2787a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 -- 172.21.15.37:0/3694337733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 msgr2=0x5637d2787a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2787a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 -- 172.21.15.37:0/3694337733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 msgr2=0x5637d2794bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 0x5637d2794bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001affd640 1 -- 172.21.15.37:0/3694337733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d23ff210 con 0x5637d2787f40 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.904+0000 7f001bfff640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 0x5637d2794bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.905+0000 7f001affd640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 0x7f00040a6300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0010014a20 tx=0x7f001000cbf0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.905+0000 7f001affd640 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00100117b0 tx=0x7f001001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c18ab621edefeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.905+0000 7f0001ffb640 1 -- 172.21.15.37:0/3694337733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001000fc60 con 0x5637d2787f40 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.905+0000 7f0001ffb640 1 -- 172.21.15.37:0/3694337733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001000fe00 con 0x5637d2787f40 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.905+0000 7f0001ffb640 1 -- 172.21.15.37:0/3694337733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100037f0 con 0x5637d2787f40 2024-04-25T21:16:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- 172.21.15.37:0/3694337733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 msgr2=0x5637d27922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f00100117b0 tx=0x7f001001b000 comp rx=0 tx=0).stop 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- 172.21.15.37:0/3694337733 shutdown_connections 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2792800 0x5637d2794bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d27922c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 --2- 172.21.15.37:0/3694337733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2787a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- 172.21.15.37:0/3694337733 >> 172.21.15.37:0/3694337733 conn(0x5637d2783b30 msgr2=0x5637d2783f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- 172.21.15.37:0/3694337733 shutdown_connections 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- 172.21.15.37:0/3694337733 wait complete. 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 -- start start 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.906+0000 7f0024ba5740 1 Processor -- start 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d278f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5637d27953d0 con 0x5637d2792800 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5637d26870b0 con 0x5637d2787620 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5637d2795550 con 0x5637d2787f40 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 10 client.0 _ll_register_callbacks cb 0x5637d2674d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39602/0 (socket says 172.21.15.37:39602) 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 -- 172.21.15.37:0/1611200405 learned_addr learned my addr 172.21.15.37:0/1611200405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001affd640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d278f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 -- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 msgr2=0x5637d278f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d278f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 -- 172.21.15.37:0/1611200405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 msgr2=0x5637d27920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 -- 172.21.15.37:0/1611200405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d24cb620 con 0x5637d2792800 2024-04-25T21:16:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f001bfff640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00140ba200 tx=0x7f00140bd700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241f545479364647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7efffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140491d0 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f00140ef630 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.907+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f001000fe00 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00140d7280 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 10 client.16311 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637d27953d0 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f001000fe00 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0014004610 con 0x5637d2792800 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 10 client.16311 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 20 client.16311 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d27a2700 0x5637d27a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 4 client.16311 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637d26fa390 con 0x5637d27a2700 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.908+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d27a2700 0x5637d27a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.909+0000 7efffaffd640 10 client.16311 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.909+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d27a2700 0x5637d27a4b60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b2c0928a38f47b server_cookie=43eb271cf6fe71a6 in_seq=0 out_seq=0 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7efffaffd640 1 -- 172.21.15.37:0/1611200405 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637d26fa390 con 0x5637d27a2700 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7efffaffd640 10 client.16311 handle_command_reply: tid=0 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 client.16311 shutdown 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 msgr2=0x5637d278f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00140ba200 tx=0x7f00140bd700 comp rx=0 tx=0).stop 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 shutdown_connections 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d27a2700 0x5637d27a4b60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2792800 0x5637d278f920 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d278f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7efffaffd640 0 client.16311 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1611200405 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 >> 172.21.15.37:0/1611200405 conn(0x5637d2783b30 msgr2=0x5637d27898b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 shutdown_connections 2024-04-25T21:16:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 1 -- 172.21.15.37:0/1611200405 wait complete. 2024-04-25T21:16:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.910+0000 7f0024ba5740 20 client.16311 trim_cache size 0 max 16384 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 Processor -- start 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 -- start start 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d27aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1afca0 con 0x5637d2783b30 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1b17e0 con 0x5637d2787620 2024-04-25T21:16:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.925+0000 7f0024ba5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1b0370 con 0x5637d2787f40 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39614/0 (socket says 172.21.15.37:39614) 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 -- 172.21.15.37:0/730885599 learned_addr learned my addr 172.21.15.37:0/730885599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001bfff640 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d27aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 -- 172.21.15.37:0/730885599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 msgr2=0x5637d27aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d27aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 -- 172.21.15.37:0/730885599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 msgr2=0x5637d27aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 -- 172.21.15.37:0/730885599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637d27a51e0 con 0x5637d2783b30 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001bfff640 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d27aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f001b7fe640 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f000c005f00 tx=0x7f000c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40058c7158a33f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f00017fa640 1 -- 172.21.15.37:0/730885599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c003d60 con 0x5637d2783b30 2024-04-25T21:16:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f00017fa640 1 -- 172.21.15.37:0/730885599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c023020 con 0x5637d2783b30 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.926+0000 7f00017fa640 1 -- 172.21.15.37:0/730885599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c011800 con 0x5637d2783b30 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- 172.21.15.37:0/730885599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 msgr2=0x5637d279e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f000c005f00 tx=0x7f000c00b390 comp rx=0 tx=0).stop 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- 172.21.15.37:0/730885599 shutdown_connections 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787f40 0x5637d27aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2787620 0x5637d27aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 --2- 172.21.15.37:0/730885599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2783b30 0x5637d279e290 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- 172.21.15.37:0/730885599 >> 172.21.15.37:0/730885599 conn(0x5637d278fa80 msgr2=0x5637d2782cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- 172.21.15.37:0/730885599 shutdown_connections 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- 172.21.15.37:0/730885599 wait complete. 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.927+0000 7f0024ba5740 1 -- start start 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 Processor -- start 2024-04-25T21:16:10.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2783b30 0x5637d2798830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2798d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1b1110 con 0x5637d2787f40 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1b0a40 con 0x5637d2783b30 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1adca0 con 0x5637d2787620 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f0024ba5740 10 client.0 _ll_register_callbacks cb 0x7f00140bdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39618/0 (socket says 172.21.15.37:39618) 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 -- 172.21.15.37:0/1392994003 learned_addr learned my addr 172.21.15.37:0/1392994003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001affd640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2798d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2783b30 0x5637d2798830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 -- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 msgr2=0x5637d2798d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2798d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 -- 172.21.15.37:0/1392994003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2783b30 msgr2=0x5637d2798830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2783b30 0x5637d2798830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 -- 172.21.15.37:0/1392994003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637d2794bf0 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001affd640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2798d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7f001bfff640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f00140049d0 tx=0x7f0014005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c71d77e7a3f51c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.928+0000 7efffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140ef770 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8557+0+0 (secure 0 0 0) 0x7f00140c64f0 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000c023020 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f00140d7280 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 10 client.16317 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637d2794bf0 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c023020 con 0x5637d2787f40 2024-04-25T21:16:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f001409cbc0 con 0x5637d2787f40 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 10 client.16317 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 20 client.16317 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d2792800 0x5637d280b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 4 client.16317 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637d26f4200 con 0x5637d2792800 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.929+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d2792800 0x5637d280b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.930+0000 7f001b7fe640 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d2792800 0x5637d280b5e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd45989e93745fb server_cookie=9b17d0830497b97f in_seq=0 out_seq=0 2024-04-25T21:16:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.930+0000 7efffaffd640 10 client.16317 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7efffaffd640 1 -- 172.21.15.37:0/1392994003 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637d26f4200 con 0x5637d2792800 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7efffaffd640 10 client.16317 handle_command_reply: tid=0 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 client.16317 shutdown 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 msgr2=0x5637d2807cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f00140049d0 tx=0x7f0014005ef0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 shutdown_connections 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637d2792800 0x5637d280b5e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5637d2787f40 0x5637d2807cf0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7efffaffd640 0 client.16317 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5637d2787620 0x5637d2798d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 --2- 172.21.15.37:0/1392994003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5637d2783b30 0x5637d2798830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 >> 172.21.15.37:0/1392994003 conn(0x5637d278fa80 msgr2=0x5637d279d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 shutdown_connections 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 1 -- 172.21.15.37:0/1392994003 wait complete. 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.939+0000 7f0024ba5740 20 client.16317 trim_cache size 0 max 16384 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:16:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:16:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 msgr2=0x7f00040a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 0x7f00040a6300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0010014a20 tx=0x7f001000cbf0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 msgr2=0x7f001c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f00140319f0 tx=0x7f0014031a20 comp rx=0 tx=0).stop 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 shutdown_connections 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f001c10e910 0x7f001c19a770 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f001c101680 0x7f001c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f00040a3e40 0x7f00040a6300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 --2- 172.21.15.37:0/2339424730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f001c100d60 0x7f001c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 >> 172.21.15.37:0/2339424730 conn(0x7f001c077fd0 msgr2=0x7f001c10fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 shutdown_connections 2024-04-25T21:16:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:10.940+0000 7f0021e8a640 1 -- 172.21.15.37:0/2339424730 wait complete. 2024-04-25T21:16:11.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:11 smithi177 ceph-mon[33043]: pgmap v237: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:11.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:11 smithi098 ceph-mon[32569]: pgmap v237: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:11.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:11 smithi037 ceph-mon[29195]: pgmap v237: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:13.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:13 smithi177 ceph-mon[33043]: pgmap v238: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:13.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:13 smithi098 ceph-mon[32569]: pgmap v238: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:13 smithi037 ceph-mon[29195]: pgmap v238: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:16:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:15 smithi177 ceph-mon[33043]: pgmap v239: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:15 smithi177 ceph-mon[33043]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:16:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:15 smithi098 ceph-mon[32569]: pgmap v239: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:15 smithi098 ceph-mon[32569]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:16:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:15 smithi037 ceph-mon[29195]: pgmap v239: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:15.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:15 smithi037 ceph-mon[29195]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:16:17.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:17 smithi177 ceph-mon[33043]: pgmap v241: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-04-25T21:16:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:17 smithi098 ceph-mon[32569]: pgmap v241: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-04-25T21:16:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:17 smithi037 ceph-mon[29195]: pgmap v241: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-04-25T21:16:19.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:19 smithi177 ceph-mon[33043]: pgmap v242: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:19.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:19 smithi098 ceph-mon[32569]: pgmap v242: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:19 smithi037 ceph-mon[29195]: pgmap v242: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-04-25T21:16:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:21 smithi177 ceph-mon[33043]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:21.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:21 smithi098 ceph-mon[32569]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:21 smithi037 ceph-mon[29195]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:23.801 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:23 smithi037 ceph-mon[29195]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:23.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:23 smithi177 ceph-mon[33043]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:23.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:23 smithi098 ceph-mon[32569]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.030+0000 7f14dfdad640 1 -- 172.21.15.37:0/3535277130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 msgr2=0x7f14d810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 --2- 172.21.15.37:0/3535277130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d810e9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009ca0 tx=0x7f14d002a9e0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 -- 172.21.15.37:0/3535277130 shutdown_connections 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 --2- 172.21.15.37:0/3535277130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d810e9c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 --2- 172.21.15.37:0/3535277130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 0x7f14d810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 --2- 172.21.15.37:0/3535277130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 0x7f14d81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 -- 172.21.15.37:0/3535277130 >> 172.21.15.37:0/3535277130 conn(0x7f14d8077fd0 msgr2=0x7f14d80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 -- 172.21.15.37:0/3535277130 shutdown_connections 2024-04-25T21:16:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 -- 172.21.15.37:0/3535277130 wait complete. 2024-04-25T21:16:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 Processor -- start 2024-04-25T21:16:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.031+0000 7f14dfdad640 1 -- start start 2024-04-25T21:16:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 0x7f14d8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 0x7f14d8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81121b0 con 0x7f14d810c520 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8112030 con 0x7f14d81031e0 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dfdad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8112330 con 0x7f14d8103b00 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37702/0 (socket says 172.21.15.37:37702) 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 -- 172.21.15.37:0/657836836 learned_addr learned my addr 172.21.15.37:0/657836836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14dd348640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 0x7f14d8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14ddb49640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 0x7f14d8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 -- 172.21.15.37:0/657836836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 msgr2=0x7f14d8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 0x7f14d8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 -- 172.21.15.37:0/657836836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 msgr2=0x7f14d8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 0x7f14d8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 -- 172.21.15.37:0/657836836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819d640 con 0x7f14d810c520 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14de34a640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009dc0 tx=0x7f14d002e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.032+0000 7f14caffd640 1 -- 172.21.15.37:0/657836836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0033040 con 0x7f14d810c520 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.033+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819d8d0 con 0x7f14d810c520 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.033+0000 7f14caffd640 1 -- 172.21.15.37:0/657836836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d0038b00 con 0x7f14d810c520 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.033+0000 7f14caffd640 1 -- 172.21.15.37:0/657836836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0043a00 con 0x7f14d810c520 2024-04-25T21:16:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.033+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a51c0 con 0x7f14d810c520 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.035+0000 7f14caffd640 1 -- 172.21.15.37:0/657836836 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14d0036070 con 0x7f14d810c520 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.035+0000 7f14caffd640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 0x7f14c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.035+0000 7f14ddb49640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 0x7f14c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.035+0000 7f14caffd640 1 -- 172.21.15.37:0/657836836 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f14d00f17f0 con 0x7f14d810c520 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.036+0000 7f14ddb49640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 0x7f14c40a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0058c0 tx=0x7f14cc00a2d0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.036+0000 7f14e2ba2740 1 Processor -- start 2024-04-25T21:16:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.036+0000 7f14e2ba2740 1 -- start start 2024-04-25T21:16:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 0x562542b012c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542b01800 0x562542b03bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562542a5b970 con 0x562542af6620 2024-04-25T21:16:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5625429f84a0 con 0x562542b01800 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562542a57930 con 0x562542af6f40 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37712/0 (socket says 172.21.15.37:37712) 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 -- 172.21.15.37:0/3505253145 learned_addr learned my addr 172.21.15.37:0/3505253145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14de34a640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542b01800 0x562542b03bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14dd348640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 0x562542b012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 -- 172.21.15.37:0/3505253145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 msgr2=0x562542b012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 0x562542b012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 -- 172.21.15.37:0/3505253145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542b01800 msgr2=0x562542b03bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542b01800 0x562542b03bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 -- 172.21.15.37:0/3505253145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56254276e210 con 0x562542af6620 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14dd348640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 0x562542b012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.037+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00ff40 tx=0x7f14cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9a6cdec05c6316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14c3fff640 1 -- 172.21.15.37:0/3505253145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016070 con 0x562542af6620 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14c3fff640 1 -- 172.21.15.37:0/3505253145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc01b8d0 con 0x562542af6620 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14c3fff640 1 -- 172.21.15.37:0/3505253145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc01c8f0 con 0x562542af6620 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3505253145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 msgr2=0x562542af6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00ff40 tx=0x7f14cc012000 comp rx=0 tx=0).stop 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3505253145 shutdown_connections 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542b01800 0x562542b03bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6f40 0x562542b012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3505253145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6620 0x562542af6a00 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3505253145 >> 172.21.15.37:0/3505253145 conn(0x562542af2b30 msgr2=0x562542af2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3505253145 shutdown_connections 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.038+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3505253145 wait complete. 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 -- start start 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 Processor -- start 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542b010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542afe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562542b04430 con 0x562542b01800 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5625429f60b0 con 0x562542af6f40 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x562542b045b0 con 0x562542af6620 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14e2ba2740 10 client.0 _ll_register_callbacks cb 0x5625429e3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37714/0 (socket says 172.21.15.37:37714) 2024-04-25T21:16:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14de34a640 1 -- 172.21.15.37:0/3728165953 learned_addr learned my addr 172.21.15.37:0/3728165953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.039+0000 7f14dd348640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542afe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542b010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 -- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 msgr2=0x562542b010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542b010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 -- 172.21.15.37:0/3728165953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 msgr2=0x562542afe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542afe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 -- 172.21.15.37:0/3728165953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56254283a620 con 0x562542b01800 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14de34a640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f14d00bd540 tx=0x7f14d002fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd812d15e186c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00c6400 con 0x562542b01800 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f14d00a80c0 con 0x562542b01800 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc01b8d0 con 0x562542b01800 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14d00db050 con 0x562542b01800 2024-04-25T21:16:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14e2ba2740 10 client.16329 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562542b04430 con 0x562542b01800 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.040+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc01b8d0 con 0x562542b01800 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14d0043ba0 con 0x562542b01800 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 10 client.16329 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 20 client.16329 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b11690 0x562542b13ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 4 client.16329 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562542a69390 con 0x562542b11690 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b11690 0x562542b13ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14ddb49640 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b11690 0x562542b13ab0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc433086cd78bdf server_cookie=d2aee63de663b085 in_seq=0 out_seq=0 2024-04-25T21:16:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.041+0000 7f14c17fa640 10 client.16329 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14c17fa640 1 -- 172.21.15.37:0/3728165953 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562542a69390 con 0x562542b11690 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14c17fa640 10 client.16329 handle_command_reply: tid=0 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14e2ba2740 1 client.16329 shutdown 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 msgr2=0x562542afe900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f14d00bd540 tx=0x7f14d002fcf0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 shutdown_connections 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.042+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b11690 0x562542b13ab0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542b01800 0x562542afe900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542afe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/3728165953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542b010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14c17fa640 0 client.16329 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 >> 172.21.15.37:0/3728165953 conn(0x562542af2b30 msgr2=0x562542af8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 shutdown_connections 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 1 -- 172.21.15.37:0/3728165953 wait complete. 2024-04-25T21:16:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.043+0000 7f14e2ba2740 20 client.16329 trim_cache size 0 max 16384 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.071+0000 7f14e2ba2740 1 Processor -- start 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.071+0000 7f14e2ba2740 1 -- start start 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542af9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542b00c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81adae0 con 0x562542af2b30 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81af620 con 0x562542af6f40 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ae1b0 con 0x562542af6620 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37730/0 (socket says 172.21.15.37:37730) 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 -- 172.21.15.37:0/1817420013 learned_addr learned my addr 172.21.15.37:0/1817420013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14de34a640 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542b00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14dd348640 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542af9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 -- 172.21.15.37:0/1817420013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 msgr2=0x562542af9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542af9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 -- 172.21.15.37:0/1817420013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 msgr2=0x562542b00c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542b00c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 -- 172.21.15.37:0/1817420013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562542af6d70 con 0x562542af2b30 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.072+0000 7f14ddb49640 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f14cc01b660 tx=0x7f14cc010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbab158ccba9c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14c37fe640 1 -- 172.21.15.37:0/1817420013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016070 con 0x562542af2b30 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14c37fe640 1 -- 172.21.15.37:0/1817420013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc034e50 con 0x562542af2b30 2024-04-25T21:16:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14c37fe640 1 -- 172.21.15.37:0/1817420013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc02a040 con 0x562542af2b30 2024-04-25T21:16:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 -- 172.21.15.37:0/1817420013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 msgr2=0x562542b0d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f14cc01b660 tx=0x7f14cc010d40 comp rx=0 tx=0).stop 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 -- 172.21.15.37:0/1817420013 shutdown_connections 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6f40 0x562542b00c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af6620 0x562542af9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/1817420013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af2b30 0x562542b0d2f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 -- 172.21.15.37:0/1817420013 >> 172.21.15.37:0/1817420013 conn(0x562542afea60 msgr2=0x562542af9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 -- 172.21.15.37:0/1817420013 shutdown_connections 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.073+0000 7f14e2ba2740 1 -- 172.21.15.37:0/1817420013 wait complete. 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 -- start start 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 Processor -- start 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 0x562542b057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81aef50 con 0x562542af6f40 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ae880 con 0x562542af6620 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81abae0 con 0x562542af2b30 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14e2ba2740 10 client.0 _ll_register_callbacks cb 0x7f14d00bd380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14de34a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37738/0 (socket says 172.21.15.37:37738) 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14ddb49640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:38770/0 (socket says 172.21.15.37:38770) 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.074+0000 7f14de34a640 1 -- 172.21.15.37:0/4097596295 learned_addr learned my addr 172.21.15.37:0/4097596295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14dd348640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 0x562542b057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 -- 172.21.15.37:0/4097596295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 msgr2=0x562542b057a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 0x562542b057a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 -- 172.21.15.37:0/4097596295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 msgr2=0x562542b05ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 -- 172.21.15.37:0/4097596295 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562542b76c80 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14dd348640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 0x562542b057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14de34a640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14ddb49640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00ff10 tx=0x7f14cc005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b91a99149e08ca22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc016040 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f14cc034540 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc034e50 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f14cc034740 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14e2ba2740 10 client.25813 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.075+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562542b77f70 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14cc034e50 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f14cc01ba90 con 0x562542af2b30 2024-04-25T21:16:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 20 client.25813 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b01800 0x562542b7a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 4 client.25813 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562542a63200 con 0x562542b01800 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.076+0000 7f14dd348640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b01800 0x562542b7a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.077+0000 7f14c17fa640 10 client.25813 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.077+0000 7f14dd348640 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b01800 0x562542b7a440 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e438863be50afbee server_cookie=927c925616e6b0a9 in_seq=0 out_seq=0 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14c17fa640 1 -- 172.21.15.37:0/4097596295 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562542a63200 con 0x562542b01800 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14c17fa640 10 client.25813 handle_command_reply: tid=0 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 client.25813 shutdown 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 msgr2=0x562542b05260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00ff10 tx=0x7f14cc005cd0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 shutdown_connections 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x562542b01800 0x562542b7a440 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562542af6f40 0x562542b05ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562542af6620 0x562542b057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 --2- 172.21.15.37:0/4097596295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562542af2b30 0x562542b05260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 >> 172.21.15.37:0/4097596295 conn(0x562542afea60 msgr2=0x562542af1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14c17fa640 0 client.25813 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 shutdown_connections 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 1 -- 172.21.15.37:0/4097596295 wait complete. 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.081+0000 7f14e2ba2740 20 client.25813 trim_cache size 0 max 16384 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:16:24.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:16:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:16:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 msgr2=0x7f14c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 0x7f14c40a6280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0058c0 tx=0x7f14cc00a2d0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 msgr2=0x7f14d819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009dc0 tx=0x7f14d002e5c0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 shutdown_connections 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14d810c520 0x7f14d819cf10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f14c40a3dc0 0x7f14c40a6280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14d8103b00 0x7f14d8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 --2- 172.21.15.37:0/657836836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d81031e0 0x7f14d8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 >> 172.21.15.37:0/657836836 conn(0x7f14d8077fd0 msgr2=0x7f14d80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 shutdown_connections 2024-04-25T21:16:24.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:24.082+0000 7f14dfdad640 1 -- 172.21.15.37:0/657836836 wait complete. 2024-04-25T21:16:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:25 smithi177 ceph-mon[33043]: pgmap v245: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T21:16:25.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:25 smithi098 ceph-mon[32569]: pgmap v245: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T21:16:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:25 smithi037 ceph-mon[29195]: pgmap v245: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-25T21:16:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:27 smithi177 ceph-mon[33043]: pgmap v246: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 377 B/s wr, 9 op/s 2024-04-25T21:16:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:27 smithi098 ceph-mon[32569]: pgmap v246: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 377 B/s wr, 9 op/s 2024-04-25T21:16:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:27 smithi037 ceph-mon[29195]: pgmap v246: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 377 B/s wr, 9 op/s 2024-04-25T21:16:28.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:16:28.576 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:16:29.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:29 smithi177 ceph-mon[33043]: pgmap v247: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:29.857 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:29 smithi037 ceph-mon[29195]: pgmap v247: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:29 smithi098 ceph-mon[32569]: pgmap v247: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 -- 172.21.15.37:0/3032167849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414101da0 msgr2=0x7fa41410f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/3032167849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414101da0 0x7fa41410f810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa40400b0a0 tx=0x7fa40400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 -- 172.21.15.37:0/3032167849 shutdown_connections 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/3032167849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414101da0 0x7fa41410f810 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/3032167849 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa414107b00 0x7fa414107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/3032167849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4141013e0 0x7fa414101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 -- 172.21.15.37:0/3032167849 >> 172.21.15.37:0/3032167849 conn(0x7fa414076c60 msgr2=0x7fa414077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:30.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 -- 172.21.15.37:0/3032167849 shutdown_connections 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.420+0000 7fa41b2ff700 1 -- 172.21.15.37:0/3032167849 wait complete. 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 Processor -- start 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 -- start start 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 0x7fa4141025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa414101da0 0x7fa414102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa414112c90 con 0x7fa414107b00 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa414112b10 con 0x7fa414101da0 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41b2ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa414112e10 con 0x7fa4141013e0 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58520/0 (socket says 172.21.15.37:58520) 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 -- 172.21.15.37:0/2537052900 learned_addr learned my addr 172.21.15.37:0/2537052900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41889a700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa414101da0 0x7fa414102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41909b700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 0x7fa4141025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 -- 172.21.15.37:0/2537052900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 msgr2=0x7fa4141025c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 0x7fa4141025c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 -- 172.21.15.37:0/2537052900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa414101da0 msgr2=0x7fa414102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa414101da0 0x7fa414102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41989c700 1 -- 172.21.15.37:0/2537052900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4141a75e0 con 0x7fa414107b00 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.421+0000 7fa41909b700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 0x7fa4141025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa41989c700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa4040062a0 tx=0x7fa404004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404021070 con 0x7fa414107b00 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa404007d00 con 0x7fa414107b00 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4141a7870 con 0x7fa414107b00 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40401d450 con 0x7fa414107b00 2024-04-25T21:16:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.422+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141a7d70 con 0x7fa414107b00 2024-04-25T21:16:30.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.423+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa414069730 con 0x7fa414107b00 2024-04-25T21:16:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.424+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa404004020 con 0x7fa414107b00 2024-04-25T21:16:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.425+0000 7fa4127fc700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 0x7fa3fc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.425+0000 7fa41909b700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 0x7fa3fc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.425+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fa40400f020 con 0x7fa414107b00 2024-04-25T21:16:30.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.425+0000 7fa41909b700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 0x7fa3fc0a62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa4140781e0 tx=0x7fa40c008040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:30.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.428+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa40402a020 con 0x7fa414107b00 2024-04-25T21:16:30.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.684+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4141a7fd0 con 0x7fa414107b00 2024-04-25T21:16:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.685+0000 7fa4127fc700 1 -- 172.21.15.37:0/2537052900 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fa40409e9e0 con 0x7fa414107b00 2024-04-25T21:16:30.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:16:30.685 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:16:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 msgr2=0x7fa3fc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 0x7fa3fc0a62f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa4140781e0 tx=0x7fa40c008040 comp rx=0 tx=0).stop 2024-04-25T21:16:30.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 msgr2=0x7fa414103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa4040062a0 tx=0x7fa404004ed0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 shutdown_connections 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa3fc0a3e30 0x7fa3fc0a62f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa414107b00 0x7fa414103040 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4141013e0 0x7fa4141025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 --2- 172.21.15.37:0/2537052900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa414101da0 0x7fa414102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 >> 172.21.15.37:0/2537052900 conn(0x7fa414076c60 msgr2=0x7fa4140ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.687+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 shutdown_connections 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:30.688+0000 7fa41b2ff700 1 -- 172.21.15.37:0/2537052900 wait complete. 2024-04-25T21:16:30.688 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:16:31.332 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 10a4283f --release --await' 2024-04-25T21:16:31.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 10a4283f --release --await 2024-04-25T21:16:31.586 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:16:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:31 smithi177 ceph-mon[33043]: pgmap v248: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T21:16:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:31 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2537052900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:16:31.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:31 smithi037 ceph-mon[29195]: pgmap v248: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T21:16:31.854 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:31 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2537052900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:16:31.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:31 smithi098 ceph-mon[32569]: pgmap v248: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-25T21:16:31.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:31 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2537052900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:16:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 -- 172.21.15.37:0/763515556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 msgr2=0x7f98cc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 --2- 172.21.15.37:0/763515556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc108e30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009990 tx=0x7f98bc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 -- 172.21.15.37:0/763515556 shutdown_connections 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 --2- 172.21.15.37:0/763515556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc103450 0x7f98cc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 --2- 172.21.15.37:0/763515556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc108e30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 --2- 172.21.15.37:0/763515556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc102a90 0x7f98cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 -- 172.21.15.37:0/763515556 >> 172.21.15.37:0/763515556 conn(0x7f98cc079ba0 msgr2=0x7f98cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 -- 172.21.15.37:0/763515556 shutdown_connections 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.268+0000 7f98d290f700 1 -- 172.21.15.37:0/763515556 wait complete. 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 Processor -- start 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 -- start start 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc102a90 0x7f98cc1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc103450 0x7f98cc199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112c60 con 0x7f98cc108a50 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112ae0 con 0x7f98cc103450 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d290f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112de0 con 0x7f98cc102a90 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d0eac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d0eac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58536/0 (socket says 172.21.15.37:58536) 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98d0eac700 1 -- 172.21.15.37:0/2536369947 learned_addr learned my addr 172.21.15.37:0/2536369947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc102a90 0x7f98cc1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.269+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc103450 0x7f98cc199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 -- 172.21.15.37:0/2536369947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc102a90 msgr2=0x7f98cc1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc102a90 0x7f98cc1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 -- 172.21.15.37:0/2536369947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc103450 msgr2=0x7f98cc199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc103450 0x7f98cc199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 -- 172.21.15.37:0/2536369947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19e0e0 con 0x7f98cc108a50 2024-04-25T21:16:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b970 tx=0x7f98c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98c97fa700 1 -- 172.21.15.37:0/2536369947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c400d860 con 0x7f98cc108a50 2024-04-25T21:16:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98c97fa700 1 -- 172.21.15.37:0/2536369947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c400e430 con 0x7f98cc108a50 2024-04-25T21:16:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19e3d0 con 0x7f98cc108a50 2024-04-25T21:16:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98c97fa700 1 -- 172.21.15.37:0/2536369947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4017660 con 0x7f98cc108a50 2024-04-25T21:16:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.270+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc074c70 con 0x7f98cc108a50 2024-04-25T21:16:33.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.272+0000 7f98c97fa700 1 -- 172.21.15.37:0/2536369947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98c400da00 con 0x7f98cc108a50 2024-04-25T21:16:33.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.273+0000 7f98c97fa700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 0x7f98b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.273+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 0x7f98b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.273+0000 7f98c97fa700 1 -- 172.21.15.37:0/2536369947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f98c40d0480 con 0x7f98cc108a50 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.273+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 0x7f98b40a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b5c0 tx=0x7f98bc005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 Processor -- start 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 -- start start 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a2f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a302260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56471a247d90 con 0x56471a2f4f70 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56471a1eea30 con 0x56471a2ffe70 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98dd198b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56471a243d50 con 0x56471a2f4650 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58542/0 (socket says 172.21.15.37:58542) 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 -- 172.21.15.37:0/435890254 learned_addr learned my addr 172.21.15.37:0/435890254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cbfff700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a2f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98d0eac700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a302260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 -- 172.21.15.37:0/435890254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 msgr2=0x56471a2f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a2f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 -- 172.21.15.37:0/435890254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 msgr2=0x56471a302260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a302260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98cb7fe700 1 -- 172.21.15.37:0/435890254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471a1b2840 con 0x56471a2f4f70 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.274+0000 7f98d0eac700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a302260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f98c000ad10 tx=0x7f98c000f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba0af8ad64aa303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98b27fc700 1 -- 172.21.15.37:0/435890254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c000ae60 con 0x56471a2f4f70 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98b27fc700 1 -- 172.21.15.37:0/435890254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c00044e0 con 0x56471a2f4f70 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98b27fc700 1 -- 172.21.15.37:0/435890254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0018690 con 0x56471a2f4f70 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98dd198b80 1 -- 172.21.15.37:0/435890254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 msgr2=0x56471a2ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.275+0000 7f98dd198b80 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f98c000ad10 tx=0x7f98c000f6a0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 -- 172.21.15.37:0/435890254 shutdown_connections 2024-04-25T21:16:33.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a302260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a2f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 --2- 172.21.15.37:0/435890254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2ff930 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 -- 172.21.15.37:0/435890254 >> 172.21.15.37:0/435890254 conn(0x56471a221990 msgr2=0x56471a2f0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 -- 172.21.15.37:0/435890254 shutdown_connections 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 -- 172.21.15.37:0/435890254 wait complete. 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 -- start start 2024-04-25T21:16:33.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.276+0000 7f98dd198b80 1 Processor -- start 2024-04-25T21:16:33.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4650 0x56471a2ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 0x56471a2fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56471a302900 con 0x56471a2f4f70 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56471a2016b0 con 0x56471a2f4650 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56471a302a80 con 0x56471a2ffe70 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 10 client.0 _ll_register_callbacks cb 0x56471a1e0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58558/0 (socket says 172.21.15.37:58558) 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 -- 172.21.15.37:0/128289709 learned_addr learned my addr 172.21.15.37:0/128289709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98d0eac700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 0x56471a2fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cbfff700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4650 0x56471a2ff5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 -- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 msgr2=0x56471a2fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 0x56471a2fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 -- 172.21.15.37:0/128289709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4650 msgr2=0x56471a2ff5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4650 0x56471a2ff5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 -- 172.21.15.37:0/128289709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471a1959d0 con 0x56471a2f4f70 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98d0eac700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 0x56471a2fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f98c00057c0 tx=0x7f98c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d25bb15dbbc5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0026020 con 0x56471a2f4f70 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f98c0004960 con 0x56471a2f4f70 2024-04-25T21:16:33.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.277+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c00044e0 con 0x56471a2f4f70 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c0031050 con 0x56471a2f4f70 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 10 client.16353 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56471a30d150 con 0x56471a2f4f70 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c00044e0 con 0x56471a2f4f70 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f98c0010430 con 0x56471a2f4f70 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 10 client.16353 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 20 client.16353 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a30f6d0 0x56471a311ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 4 client.16353 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56471a255580 con 0x56471a30f6d0 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.278+0000 7f98cbfff700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a30f6d0 0x56471a311ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.279+0000 7f98cbfff700 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a30f6d0 0x56471a311ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d018f334ee478a server_cookie=ba965184db2d6c07 in_seq=0 out_seq=0 2024-04-25T21:16:33.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.279+0000 7f98ab7fe700 10 client.16353 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98ab7fe700 1 -- 172.21.15.37:0/128289709 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56471a255580 con 0x56471a30f6d0 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98ab7fe700 10 client.16353 handle_command_reply: tid=0 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 client.16353 shutdown 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 msgr2=0x56471a2fc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f98c00057c0 tx=0x7f98c00057f0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 shutdown_connections 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a30f6d0 0x56471a311ac0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2ffe70 0x56471a2fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4650 0x56471a2ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 --2- 172.21.15.37:0/128289709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4f70 0x56471a2fc910 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 >> 172.21.15.37:0/128289709 conn(0x56471a221990 msgr2=0x56471a2f6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98ab7fe700 0 client.16353 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:33.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 shutdown_connections 2024-04-25T21:16:33.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 1 -- 172.21.15.37:0/128289709 wait complete. 2024-04-25T21:16:33.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.280+0000 7f98dd198b80 20 client.16353 trim_cache size 0 max 16384 2024-04-25T21:16:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.299+0000 7f98dd198b80 1 Processor -- start 2024-04-25T21:16:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.299+0000 7f98dd198b80 1 -- start start 2024-04-25T21:16:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4f70 0x56471a317600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a307d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1af540 con 0x56471a2f4650 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b1080 con 0x56471a2ffe70 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98dd198b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1afc10 con 0x56471a2f4f70 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58566/0 (socket says 172.21.15.37:58566) 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 -- 172.21.15.37:0/268919604 learned_addr learned my addr 172.21.15.37:0/268919604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4f70 0x56471a317600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 -- 172.21.15.37:0/268919604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4f70 msgr2=0x56471a317600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4f70 0x56471a317600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 -- 172.21.15.37:0/268919604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 msgr2=0x56471a307d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a307d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 -- 172.21.15.37:0/268919604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56471a2f16e0 con 0x56471a2f4650 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98cbfff700 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000c00 tx=0x7f98bc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b631b66e1bdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98b1ffb700 1 -- 172.21.15.37:0/268919604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc024a60 con 0x56471a2f4650 2024-04-25T21:16:33.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98b1ffb700 1 -- 172.21.15.37:0/268919604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc039070 con 0x56471a2f4650 2024-04-25T21:16:33.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.300+0000 7f98b1ffb700 1 -- 172.21.15.37:0/268919604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00fe40 con 0x56471a2f4650 2024-04-25T21:16:33.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 -- 172.21.15.37:0/268919604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 msgr2=0x56471a3170c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000c00 tx=0x7f98bc023000 comp rx=0 tx=0).stop 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 -- 172.21.15.37:0/268919604 shutdown_connections 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2ffe70 0x56471a307d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2f4650 0x56471a3170c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 --2- 172.21.15.37:0/268919604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4f70 0x56471a317600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 -- 172.21.15.37:0/268919604 >> 172.21.15.37:0/268919604 conn(0x56471a221990 msgr2=0x56471a2feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 -- 172.21.15.37:0/268919604 shutdown_connections 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.301+0000 7f98dd198b80 1 -- 172.21.15.37:0/268919604 wait complete. 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 -- start start 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 Processor -- start 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a306000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4f70 0x56471a306540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b09b0 con 0x56471a2ffe70 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1b02e0 con 0x56471a2f4f70 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc1ad500 con 0x56471a2f4650 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98dd198b80 10 client.0 _ll_register_callbacks cb 0x56471a1e0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98d0eac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98d0eac700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58582/0 (socket says 172.21.15.37:58582) 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.302+0000 7f98d0eac700 1 -- 172.21.15.37:0/2581514922 learned_addr learned my addr 172.21.15.37:0/2581514922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a306000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98cb7fe700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4f70 0x56471a306540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 -- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 msgr2=0x56471a306000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a306000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 -- 172.21.15.37:0/2581514922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4f70 msgr2=0x56471a306540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4f70 0x56471a306540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 -- 172.21.15.37:0/2581514922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56471a36efc0 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a306000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98d0eac700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98c400bfd0 tx=0x7f98c409cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb49e670d45840a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c40b0e60 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f98c409c620 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98bc039070 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f98c409c820 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98dd198b80 10 client.16359 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56471a3763d0 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.303+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc039070 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f98c401f030 con 0x56471a2ffe70 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 10 client.16359 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 10 client.16359 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:16:33.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 20 client.16359 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a376a80 0x56471a378ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 4 client.16359 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "10a4283f", "release": true, "await": true}] 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "10a4283f", "release": true, "await": true}) -- 0x56471a24f620 con 0x56471a376a80 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.304+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a376a80 0x56471a378ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.305+0000 7f98cbfff700 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a376a80 0x56471a378ea0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28a08965e4e9565 server_cookie=1e1ea526e35f12d2 in_seq=0 out_seq=0 2024-04-25T21:16:33.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.305+0000 7f98ab7fe700 10 client.16359 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:33.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98ab7fe700 1 -- 172.21.15.37:0/2581514922 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56471a24f620 con 0x56471a376a80 2024-04-25T21:16:33.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98ab7fe700 10 client.16359 handle_command_reply: tid=0 2024-04-25T21:16:33.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 client.16359 shutdown 2024-04-25T21:16:33.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 msgr2=0x56471a36e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98c400bfd0 tx=0x7f98c409cbe0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 shutdown_connections 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56471a376a80 0x56471a378ea0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56471a2ffe70 0x56471a36e980 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56471a2f4650 0x56471a306000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98ab7fe700 0 client.16359 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 --2- 172.21.15.37:0/2581514922 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56471a2f4f70 0x56471a306540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 >> 172.21.15.37:0/2581514922 conn(0x56471a221990 msgr2=0x56471a30b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 shutdown_connections 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 1 -- 172.21.15.37:0/2581514922 wait complete. 2024-04-25T21:16:33.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.309+0000 7f98dd198b80 20 client.16359 trim_cache size 0 max 16384 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 24, 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "10a4283f": { 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 24, 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:16:33.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 89.8, 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:16:33.312 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:16:33.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 msgr2=0x7f98b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 0x7f98b40a6270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b5c0 tx=0x7f98bc005960 comp rx=0 tx=0).stop 2024-04-25T21:16:33.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 msgr2=0x7f98cc19d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:33.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b970 tx=0x7f98c400bc80 comp rx=0 tx=0).stop 2024-04-25T21:16:33.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 shutdown_connections 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f98b40a3db0 0x7f98b40a6270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98cc108a50 0x7f98cc19d9a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98cc102a90 0x7f98cc1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 --2- 172.21.15.37:0/2536369947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98cc103450 0x7f98cc199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 >> 172.21.15.37:0/2536369947 conn(0x7f98cc079ba0 msgr2=0x7f98cc101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:33.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 shutdown_connections 2024-04-25T21:16:33.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:16:33.310+0000 7f98d290f700 1 -- 172.21.15.37:0/2536369947 wait complete. 2024-04-25T21:16:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:33 smithi177 ceph-mon[33043]: pgmap v249: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:33 smithi098 ceph-mon[32569]: pgmap v249: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:33 smithi037 ceph-mon[29195]: pgmap v249: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-25T21:16:34.036 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '10a4283f', seconds elapsed: 41.3 2024-04-25T21:16:35.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:35 smithi177 ceph-mon[33043]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-25T21:16:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:35 smithi098 ceph-mon[32569]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-25T21:16:35.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:35 smithi037 ceph-mon[29195]: pgmap v250: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-25T21:16:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 -- 172.21.15.37:0/1780704423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406b3a0 msgr2=0x7f81b4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 --2- 172.21.15.37:0/1780704423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406b3a0 0x7f81b4106b80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a802a410 comp rx=0 tx=0).stop 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 -- 172.21.15.37:0/1780704423 shutdown_connections 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 --2- 172.21.15.37:0/1780704423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 --2- 172.21.15.37:0/1780704423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406b3a0 0x7f81b4106b80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 --2- 172.21.15.37:0/1780704423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b406a9f0 0x7f81b406add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 -- 172.21.15.37:0/1780704423 >> 172.21.15.37:0/1780704423 conn(0x7f81b4076c50 msgr2=0x7f81b4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 -- 172.21.15.37:0/1780704423 shutdown_connections 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.182+0000 7f81ba493640 1 -- 172.21.15.37:0/1780704423 wait complete. 2024-04-25T21:16:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 Processor -- start 2024-04-25T21:16:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 -- start start 2024-04-25T21:16:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b419cf40 0x7f81b419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41121a0 con 0x7f81b406a9f0 2024-04-25T21:16:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112020 con 0x7f81b4107260 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81ba493640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81b4112320 con 0x7f81b419cf40 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b3fff640 1 -- 172.21.15.37:0/4216320428 learned_addr learned my addr 172.21.15.37:0/4216320428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b37fe640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b8a30640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b419cf40 0x7f81b419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.183+0000 7f81b3fff640 1 -- 172.21.15.37:0/4216320428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b419cf40 msgr2=0x7f81b419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b419cf40 0x7f81b419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b3fff640 1 -- 172.21.15.37:0/4216320428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 msgr2=0x7f81b4198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b4198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b3fff640 1 -- 172.21.15.37:0/4216320428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b419da30 con 0x7f81b406a9f0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b37fe640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d950 tx=0x7f81a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b17fa640 1 -- 172.21.15.37:0/4216320428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4017070 con 0x7f81b406a9f0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b419dd20 con 0x7f81b406a9f0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b17fa640 1 -- 172.21.15.37:0/4216320428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a4009c30 con 0x7f81b406a9f0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b41a5530 con 0x7f81b406a9f0 2024-04-25T21:16:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.184+0000 7f81b17fa640 1 -- 172.21.15.37:0/4216320428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a4020060 con 0x7f81b406a9f0 2024-04-25T21:16:37.194 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.186+0000 7f81b17fa640 1 -- 172.21.15.37:0/4216320428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81a4010460 con 0x7f81b406a9f0 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 Processor -- start 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 -- start start 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b17fa640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 0x7f819c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x5566440f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644105000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556644058970 con 0x5566440f3e40 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556643ff54a0 con 0x556644102c10 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556644054930 con 0x5566440f3520 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b17fa640 1 -- 172.21.15.37:0/4216320428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f81a40d08e0 con 0x7f81b406a9f0 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b8a30640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 0x7f819c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58604/0 (socket says 172.21.15.37:58604) 2024-04-25T21:16:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b37fe640 1 -- 172.21.15.37:0/2782758575 learned_addr learned my addr 172.21.15.37:0/2782758575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.187+0000 7f81b3fff640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x5566440f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 -- 172.21.15.37:0/2782758575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 msgr2=0x5566440f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x5566440f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b8a30640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 -- 172.21.15.37:0/2782758575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 msgr2=0x556644105000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644105000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 -- 172.21.15.37:0/2782758575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556643d6b210 con 0x5566440f3e40 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b8a30640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b8a30640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 0x7f819c0a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009370 tx=0x7f81ac009880 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f81b406a590 tx=0x7f81a8032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8381ba1765e6b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f819a7fc640 1 -- 172.21.15.37:0/2782758575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a803f070 con 0x5566440f3e40 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f819a7fc640 1 -- 172.21.15.37:0/2782758575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a80316a0 con 0x5566440f3e40 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.188+0000 7f819a7fc640 1 -- 172.21.15.37:0/2782758575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8035070 con 0x5566440f3e40 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2782758575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 msgr2=0x5566441026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f81b406a590 tx=0x7f81a8032cd0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2782758575 shutdown_connections 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566441026d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2782758575 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x5566440f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2782758575 >> 172.21.15.37:0/2782758575 conn(0x5566440efb30 msgr2=0x5566440eff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2782758575 shutdown_connections 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2782758575 wait complete. 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 -- start start 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.189+0000 7f81bd1a8740 1 Processor -- start 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644102560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x5566440ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566441057e0 con 0x5566440f3e40 2024-04-25T21:16:37.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556643ff30b0 con 0x556644102c10 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556644105960 con 0x5566440f3520 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81bd1a8740 10 client.0 _ll_register_callbacks cb 0x556643fe0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58618/0 (socket says 172.21.15.37:58618) 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 -- 172.21.15.37:0/4142567663 learned_addr learned my addr 172.21.15.37:0/4142567663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b8a30640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x5566440ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 -- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 msgr2=0x556644102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 -- 172.21.15.37:0/4142567663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 msgr2=0x5566440ffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x5566440ffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.190+0000 7f81b37fe640 1 -- 172.21.15.37:0/4142567663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556643e37620 con 0x5566440f3e40 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81b37fe640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f81a80095d0 tx=0x7f81a8032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b97abd816c7d40a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a804a020 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f81a8004450 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81a80316a0 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81a8055050 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556644111700 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a80316a0 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f81a80049c0 con 0x5566440f3e40 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 20 client.16371 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556644112a50 0x556644114e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 4 client.16371 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.191+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556644066390 con 0x556644112a50 2024-04-25T21:16:37.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.192+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556644112a50 0x556644114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.192+0000 7f81937fe640 10 client.16371 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.192+0000 7f81b3fff640 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556644112a50 0x556644114e40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d35d188fbc912cf server_cookie=54ef3cd2f948ebb6 in_seq=0 out_seq=0 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81937fe640 1 -- 172.21.15.37:0/4142567663 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556644066390 con 0x556644112a50 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81937fe640 10 client.16371 handle_command_reply: tid=0 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 client.16371 shutdown 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 msgr2=0x5566440ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f81a80095d0 tx=0x7f81a8032ec0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 shutdown_connections 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556644112a50 0x556644114e40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x5566440ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x5566440ff850 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81937fe640 0 client.16371 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/4142567663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 >> 172.21.15.37:0/4142567663 conn(0x5566440efb30 msgr2=0x5566440f59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 shutdown_connections 2024-04-25T21:16:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 1 -- 172.21.15.37:0/4142567663 wait complete. 2024-04-25T21:16:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.193+0000 7f81bd1a8740 20 client.16371 trim_cache size 0 max 16384 2024-04-25T21:16:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.208+0000 7f81bd1a8740 1 Processor -- start 2024-04-25T21:16:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.208+0000 7f81bd1a8740 1 -- start start 2024-04-25T21:16:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566440f3520 0x5566440faf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 0x5566440f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41ab7e0 con 0x556644102c10 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41ad320 con 0x5566440f3520 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41abeb0 con 0x5566440f3e40 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58620/0 (socket says 172.21.15.37:58620) 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 -- 172.21.15.37:0/3492142722 learned_addr learned my addr 172.21.15.37:0/3492142722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b3fff640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566440f3520 0x5566440faf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b37fe640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 0x5566440f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 -- 172.21.15.37:0/3492142722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 msgr2=0x5566440f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 0x5566440f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 -- 172.21.15.37:0/3492142722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566440f3520 msgr2=0x5566440faf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566440f3520 0x5566440faf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 -- 172.21.15.37:0/3492142722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566440f45b0 con 0x556644102c10 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b37fe640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 0x5566440f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f81b8a30640 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218c3685f26eb8cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f8199ffb640 1 -- 172.21.15.37:0/3492142722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac01f940 con 0x556644102c10 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f8199ffb640 1 -- 172.21.15.37:0/3492142722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac020430 con 0x556644102c10 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.209+0000 7f8199ffb640 1 -- 172.21.15.37:0/3492142722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ac028630 con 0x556644102c10 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- 172.21.15.37:0/3492142722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 msgr2=0x556644102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f81ac005fd0 tx=0x7f81ac014000 comp rx=0 tx=0).stop 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- 172.21.15.37:0/3492142722 shutdown_connections 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556644102c10 0x556644102090 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3e40 0x5566440f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/3492142722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5566440f3520 0x5566440faf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- 172.21.15.37:0/3492142722 >> 172.21.15.37:0/3492142722 conn(0x5566440efb30 msgr2=0x5566440f5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- 172.21.15.37:0/3492142722 shutdown_connections 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- 172.21.15.37:0/3492142722 wait complete. 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.210+0000 7f81bd1a8740 1 -- start start 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 Processor -- start 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644106510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644106f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41acc50 con 0x5566440f3e40 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41ac580 con 0x556644102c10 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81b41a9800 con 0x5566440f3520 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81bd1a8740 10 client.0 _ll_register_callbacks cb 0x7f81a8009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58628/0 (socket says 172.21.15.37:58628) 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 -- 172.21.15.37:0/2783550711 learned_addr learned my addr 172.21.15.37:0/2783550711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b8a30640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b3fff640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644106510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 -- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 msgr2=0x556644106510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644106510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 -- 172.21.15.37:0/2783550711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 msgr2=0x556644106f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644106f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.211+0000 7f81b37fe640 1 -- 172.21.15.37:0/2783550711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556644177d40 con 0x5566440f3e40 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81b37fe640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f81a802a930 tx=0x7f81a8032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5b36337d28a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a804a070 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f81a8031a60 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81ac020430 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81a803f070 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 10 client.16377 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556644178f00 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81ac020430 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f81a80035f0 con 0x5566440f3e40 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 10 client.16377 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 20 client.16377 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5566441795f0 0x55664417ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 4 client.16377 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:16:37.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.212+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556644060200 con 0x5566441795f0 2024-04-25T21:16:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.213+0000 7f81b3fff640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5566441795f0 0x55664417ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.213+0000 7f81937fe640 10 client.16377 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:37.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.213+0000 7f81b3fff640 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5566441795f0 0x55664417ba10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3320c8d08fba46 server_cookie=9819fae2c2e7162e in_seq=0 out_seq=0 2024-04-25T21:16:37.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81937fe640 1 -- 172.21.15.37:0/2783550711 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556644060200 con 0x5566441795f0 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81937fe640 10 client.16377 handle_command_reply: tid=0 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 client.16377 shutdown 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 msgr2=0x556644106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f81a802a930 tx=0x7f81a8032fc0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 shutdown_connections 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5566441795f0 0x55664417ba10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556644102c10 0x556644106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566440f3e40 0x556644106a50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81937fe640 0 client.16377 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 --2- 172.21.15.37:0/2783550711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5566440f3520 0x556644106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 >> 172.21.15.37:0/2783550711 conn(0x5566440efb30 msgr2=0x5566440fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 shutdown_connections 2024-04-25T21:16:37.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 1 -- 172.21.15.37:0/2783550711 wait complete. 2024-04-25T21:16:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.223+0000 7f81bd1a8740 20 client.16377 trim_cache size 0 max 16384 2024-04-25T21:16:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:16:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:16:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:16:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 msgr2=0x7f819c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 0x7f819c0a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f81ac009370 tx=0x7f81ac009880 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 msgr2=0x7f81b4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d950 tx=0x7f81a400de10 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 shutdown_connections 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f819c0a3dc0 0x7f819c0a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81b419cf40 0x7f81b419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81b4107260 0x7f81b4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 --2- 172.21.15.37:0/4216320428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81b406a9f0 0x7f81b4198680 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 >> 172.21.15.37:0/4216320428 conn(0x7f81b4076c50 msgr2=0x7f81b40ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:37.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 shutdown_connections 2024-04-25T21:16:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:37.224+0000 7f81ba493640 1 -- 172.21.15.37:0/4216320428 wait complete. 2024-04-25T21:16:37.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:37 smithi177 ceph-mon[33043]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:37 smithi098 ceph-mon[32569]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:37.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:37 smithi037 ceph-mon[29195]: pgmap v251: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:39 smithi177 ceph-mon[33043]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:39.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:39 smithi098 ceph-mon[32569]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:39 smithi037 ceph-mon[29195]: pgmap v252: 97 pgs: 97 active+clean; 4.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T21:16:41.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:41 smithi037 ceph-mon[29195]: pgmap v253: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:16:41.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:16:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:16:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:16:41.735 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:16:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:41 smithi177 ceph-mon[33043]: pgmap v253: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:16:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:16:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:16:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:16:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:16:41.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:41 smithi098 ceph-mon[32569]: pgmap v253: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:16:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:16:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:16:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:16:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:16:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:43 smithi177 ceph-mon[33043]: pgmap v254: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T21:16:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:43 smithi098 ceph-mon[32569]: pgmap v254: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T21:16:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:43 smithi037 ceph-mon[29195]: pgmap v254: 97 pgs: 97 active+clean; 4.8 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-25T21:16:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:45 smithi177 ceph-mon[33043]: pgmap v255: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T21:16:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:45 smithi098 ceph-mon[32569]: pgmap v255: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T21:16:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:45 smithi037 ceph-mon[29195]: pgmap v255: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 19 op/s 2024-04-25T21:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:47 smithi177 ceph-mon[33043]: pgmap v256: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:47.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:47 smithi098 ceph-mon[32569]: pgmap v256: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:47 smithi037 ceph-mon[29195]: pgmap v256: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:49 smithi098 ceph-mon[32569]: pgmap v257: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:49 smithi037 ceph-mon[29195]: pgmap v257: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:49 smithi177 ceph-mon[33043]: pgmap v257: 97 pgs: 97 active+clean; 5.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 14 op/s 2024-04-25T21:16:50.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 -- 172.21.15.37:0/3811672032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b40102b80 msgr2=0x7f5b40106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 --2- 172.21.15.37:0/3811672032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b40102b80 0x7f5b40106bc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009900 tx=0x7f5b3402a410 comp rx=0 tx=0).stop 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 -- 172.21.15.37:0/3811672032 shutdown_connections 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 --2- 172.21.15.37:0/3811672032 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 0x7f5b4010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 --2- 172.21.15.37:0/3811672032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b40102b80 0x7f5b40106bc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 --2- 172.21.15.37:0/3811672032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b401021d0 0x7f5b401025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 -- 172.21.15.37:0/3811672032 >> 172.21.15.37:0/3811672032 conn(0x7f5b400793e0 msgr2=0x7f5b400797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 -- 172.21.15.37:0/3811672032 shutdown_connections 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.326+0000 7f5b45842640 1 -- 172.21.15.37:0/3811672032 wait complete. 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 Processor -- start 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 -- start start 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 0x7f5b401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 0x7f5b4019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401121e0 con 0x7f5b401021d0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40112060 con 0x7f5b401072a0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b45842640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b40112360 con 0x7f5b4019a6e0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53230/0 (socket says 172.21.15.37:53230) 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b3effd640 1 -- 172.21.15.37:0/2856038065 learned_addr learned my addr 172.21.15.37:0/2856038065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.327+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 0x7f5b401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 0x7f5b4019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 -- 172.21.15.37:0/2856038065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 msgr2=0x7f5b4019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 0x7f5b4019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 -- 172.21.15.37:0/2856038065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 msgr2=0x7f5b401a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 0x7f5b401a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 -- 172.21.15.37:0/2856038065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4019b2e0 con 0x7f5b401021d0 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 0x7f5b4019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:50.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d770 tx=0x7f5b3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b44840640 1 -- 172.21.15.37:0/2856038065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b401021d0 2024-04-25T21:16:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b44840640 1 -- 172.21.15.37:0/2856038065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b300054d0 con 0x7f5b401021d0 2024-04-25T21:16:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b401a73c0 con 0x7f5b401021d0 2024-04-25T21:16:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b44840640 1 -- 172.21.15.37:0/2856038065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30004c30 con 0x7f5b401021d0 2024-04-25T21:16:50.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.328+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b401a7770 con 0x7f5b401021d0 2024-04-25T21:16:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b44840640 1 -- 172.21.15.37:0/2856038065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b30004e10 con 0x7f5b401021d0 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 Processor -- start 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 -- start start 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 0x559f2c443a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c44e800 0x559f2c450bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f2c3a8970 con 0x559f2c443f40 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559f2c3454a0 con 0x559f2c443620 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b48630740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559f2c3a4930 con 0x559f2c44e800 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b44840640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 0x7f5b280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53236/0 (socket says 172.21.15.37:53236) 2024-04-25T21:16:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b3effd640 1 -- 172.21.15.37:0/2557672661 learned_addr learned my addr 172.21.15.37:0/2557672661 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c44e800 0x559f2c450bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.331+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 0x559f2c443a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 0x7f5b280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b44840640 1 -- 172.21.15.37:0/2856038065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5b300d02d0 con 0x7f5b401021d0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 -- 172.21.15.37:0/2557672661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c44e800 msgr2=0x559f2c450bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c44e800 0x559f2c450bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 -- 172.21.15.37:0/2557672661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 msgr2=0x559f2c443a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 0x559f2c443a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 -- 172.21.15.37:0/2557672661 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2c0bb210 con 0x559f2c443f40 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 0x559f2c443a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3effd640 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5b300cda80 tx=0x7f5b30009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a4c966b6b55555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 0x7f5b280a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009fd0 tx=0x7f5b38009280 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b25ffb640 1 -- 172.21.15.37:0/2557672661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300cf020 con 0x559f2c443f40 2024-04-25T21:16:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b25ffb640 1 -- 172.21.15.37:0/2557672661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b300a0620 con 0x559f2c443f40 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.332+0000 7f5b25ffb640 1 -- 172.21.15.37:0/2557672661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30020070 con 0x559f2c443f40 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- 172.21.15.37:0/2557672661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 msgr2=0x559f2c44e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5b300cda80 tx=0x7f5b30009f90 comp rx=0 tx=0).stop 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- 172.21.15.37:0/2557672661 shutdown_connections 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c44e800 0x559f2c450bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443f40 0x559f2c44e2c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 --2- 172.21.15.37:0/2557672661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443620 0x559f2c443a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- 172.21.15.37:0/2557672661 >> 172.21.15.37:0/2557672661 conn(0x559f2c43fb30 msgr2=0x559f2c43ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- 172.21.15.37:0/2557672661 shutdown_connections 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- 172.21.15.37:0/2557672661 wait complete. 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.333+0000 7f5b48630740 1 -- start start 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 Processor -- start 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c44b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c44b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f2c4513d0 con 0x559f2c443620 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559f2c3430b0 con 0x559f2c44e800 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559f2c451550 con 0x559f2c443f40 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b48630740 10 client.0 _ll_register_callbacks cb 0x559f2c330d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53240/0 (socket says 172.21.15.37:53240) 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 -- 172.21.15.37:0/1182621536 learned_addr learned my addr 172.21.15.37:0/1182621536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c44b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c44b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 -- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 msgr2=0x559f2c44b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c44b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 -- 172.21.15.37:0/1182621536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 msgr2=0x559f2c44b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.334+0000 7f5b3effd640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c44b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b3effd640 1 -- 172.21.15.37:0/1182621536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2c187620 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c44b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b3effd640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ce020 tx=0x7f5b3009c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeaa82c5be6d368 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300cf050 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5b300a01b0 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b300a0620 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b30020070 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b48630740 10 client.16389 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f2c45d2b0 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b300a0620 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b30097fc0 con 0x559f2c443620 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.335+0000 7f5b48630740 10 client.16389 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b48630740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b48630740 20 client.16389 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c45e600 0x559f2c4609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b48630740 4 client.16389 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f2c3b6390 con 0x559f2c45e600 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c45e600 0x559f2c4609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c45e600 0x559f2c4609f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb2d6fee3ba17a2 server_cookie=204bca189e539902 in_seq=0 out_seq=0 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.336+0000 7f5b1effd640 10 client.16389 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b1effd640 1 -- 172.21.15.37:0/1182621536 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f2c3b6390 con 0x559f2c45e600 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b1effd640 10 client.16389 handle_command_reply: tid=0 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 client.16389 shutdown 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 msgr2=0x559f2c44e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b300ce020 tx=0x7f5b3009c170 comp rx=0 tx=0).stop 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 shutdown_connections 2024-04-25T21:16:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c45e600 0x559f2c4609f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c44b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c44b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 --2- 172.21.15.37:0/1182621536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c44e120 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b1effd640 0 client.16389 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 >> 172.21.15.37:0/1182621536 conn(0x559f2c43fb30 msgr2=0x559f2c4458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 shutdown_connections 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 1 -- 172.21.15.37:0/1182621536 wait complete. 2024-04-25T21:16:50.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.337+0000 7f5b48630740 20 client.16389 trim_cache size 0 max 16384 2024-04-25T21:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.352+0000 7f5b48630740 1 Processor -- start 2024-04-25T21:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.352+0000 7f5b48630740 1 -- start start 2024-04-25T21:16:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443620 0x559f2c44a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443f40 0x559f2c44dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401afca0 con 0x559f2c44e800 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b17e0 con 0x559f2c443f40 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b48630740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b0370 con 0x559f2c443620 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53244/0 (socket says 172.21.15.37:53244) 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 -- 172.21.15.37:0/3076069640 learned_addr learned my addr 172.21.15.37:0/3076069640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443f40 0x559f2c44dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3effd640 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443620 0x559f2c44a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 -- 172.21.15.37:0/3076069640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443620 msgr2=0x559f2c44a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443620 0x559f2c44a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 -- 172.21.15.37:0/3076069640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443f40 msgr2=0x559f2c44dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443f40 0x559f2c44dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 -- 172.21.15.37:0/3076069640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2c443d70 con 0x559f2c44e800 2024-04-25T21:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b38012840 tx=0x7f5b380097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef1c209e82eff59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.353+0000 7f5b257fa640 1 -- 172.21.15.37:0/3076069640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38012de0 con 0x559f2c44e800 2024-04-25T21:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b257fa640 1 -- 172.21.15.37:0/3076069640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b380109d0 con 0x559f2c44e800 2024-04-25T21:16:50.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b257fa640 1 -- 172.21.15.37:0/3076069640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380055b0 con 0x559f2c44e800 2024-04-25T21:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- 172.21.15.37:0/3076069640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 msgr2=0x559f2c465ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5b38012840 tx=0x7f5b380097a0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- 172.21.15.37:0/3076069640 shutdown_connections 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c44e800 0x559f2c465ff0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c443f40 0x559f2c44dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 --2- 172.21.15.37:0/3076069640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443620 0x559f2c44a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- 172.21.15.37:0/3076069640 >> 172.21.15.37:0/3076069640 conn(0x559f2c43fb30 msgr2=0x559f2c449bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- 172.21.15.37:0/3076069640 shutdown_connections 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- 172.21.15.37:0/3076069640 wait complete. 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.354+0000 7f5b48630740 1 -- start start 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 Processor -- start 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c453f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c4bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b1110 con 0x559f2c443620 2024-04-25T21:16:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401b0a40 con 0x559f2c44e800 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f5b401adca0 con 0x559f2c443f40 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b48630740 10 client.0 _ll_register_callbacks cb 0x7f5b300046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53250/0 (socket says 172.21.15.37:53250) 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 -- 172.21.15.37:0/3968733092 learned_addr learned my addr 172.21.15.37:0/3968733092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c453f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c4bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 -- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 msgr2=0x559f2c453f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c453f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 -- 172.21.15.37:0/3968733092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 msgr2=0x559f2c4bc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c4bc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3effd640 1 -- 172.21.15.37:0/3968733092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2c4bce20 con 0x559f2c443620 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.355+0000 7f5b3f7fe640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c4bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b3effd640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5b30098c40 tx=0x7f5b30098580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea180827f745b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:16:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b300cf070 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7f5b300a01b0 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b380109d0 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b300a0620 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 10 client.16395 fetch_fsmap learned FSMap version 27 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f2c4c41b0 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b380109d0 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b3009b130 con 0x559f2c443620 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 10 client.16395 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 20 client.16395 populate_metadata read hostname 'smithi037' 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c4c4570 0x559f2c4c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:16:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 4 client.16395 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.356+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f2c3b0200 con 0x559f2c4c4570 2024-04-25T21:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.357+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c4c4570 0x559f2c4c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.357+0000 7f5b1effd640 10 client.16395 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.357+0000 7f5b3e7fc640 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c4c4570 0x559f2c4c69b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30a41156046c04 server_cookie=aca92122f4db3b6a in_seq=0 out_seq=0 2024-04-25T21:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b1effd640 1 -- 172.21.15.37:0/3968733092 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f2c3b0200 con 0x559f2c4c4570 2024-04-25T21:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b1effd640 10 client.16395 handle_command_reply: tid=0 2024-04-25T21:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 client.16395 shutdown 2024-04-25T21:16:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 msgr2=0x559f2c453a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5b30098c40 tx=0x7f5b30098580 comp rx=0 tx=0).stop 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 shutdown_connections 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559f2c4c4570 0x559f2c4c69b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559f2c44e800 0x559f2c4bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b1effd640 0 client.16395 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559f2c443f40 0x559f2c453f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 --2- 172.21.15.37:0/3968733092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2c443620 0x559f2c453a40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 >> 172.21.15.37:0/3968733092 conn(0x559f2c43fb30 msgr2=0x559f2c44a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 shutdown_connections 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 1 -- 172.21.15.37:0/3968733092 wait complete. 2024-04-25T21:16:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.372+0000 7f5b48630740 20 client.16395 trim_cache size 0 max 16384 2024-04-25T21:16:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:16:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:16:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:16:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:16:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 msgr2=0x7f5b280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 0x7f5b280a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009fd0 tx=0x7f5b38009280 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 msgr2=0x7f5b401a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d770 tx=0x7f5b3000dc30 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 shutdown_connections 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f5b280a3dc0 0x7f5b280a6280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5b4019a6e0 0x7f5b4019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5b401072a0 0x7f5b401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 --2- 172.21.15.37:0/2856038065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b401021d0 0x7f5b401a0620 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 >> 172.21.15.37:0/2856038065 conn(0x7f5b400793e0 msgr2=0x7f5b401000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 shutdown_connections 2024-04-25T21:16:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:16:50.373+0000 7f5b45842640 1 -- 172.21.15.37:0/2856038065 wait complete. 2024-04-25T21:16:51.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:51 smithi098 ceph-mon[32569]: pgmap v258: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:16:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:51 smithi037 ceph-mon[29195]: pgmap v258: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:16:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:51 smithi177 ceph-mon[33043]: pgmap v258: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:16:53.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:53 smithi098 ceph-mon[32569]: pgmap v259: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:16:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:53 smithi037 ceph-mon[29195]: pgmap v259: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:16:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:53 smithi177 ceph-mon[33043]: pgmap v259: 97 pgs: 97 active+clean; 5.2 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:16:55.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:55 smithi098 ceph-mon[32569]: pgmap v260: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:16:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:55 smithi037 ceph-mon[29195]: pgmap v260: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:16:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:55 smithi177 ceph-mon[33043]: pgmap v260: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:16:57.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:57 smithi098 ceph-mon[32569]: pgmap v261: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:16:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:57 smithi037 ceph-mon[29195]: pgmap v261: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:16:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:57 smithi177 ceph-mon[33043]: pgmap v261: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:16:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:16:59 smithi098 ceph-mon[32569]: pgmap v262: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:16:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:16:59 smithi037 ceph-mon[29195]: pgmap v262: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:16:59 smithi177 ceph-mon[33043]: pgmap v262: 97 pgs: 97 active+clean; 5.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:01.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:01 smithi098 ceph-mon[32569]: pgmap v263: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:17:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:01 smithi037 ceph-mon[29195]: pgmap v263: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:17:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:01 smithi177 ceph-mon[33043]: pgmap v263: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-25T21:17:02.547 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:189/600: open fd = 3 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:190/600: open fd = 3 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:191/600: open fd = 3 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.548 INFO:tasks.workunit.client.0.smithi037.stdout:192/600: open fd = 3 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:193/600: open fd = 3 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:194/600: open fd = 3 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:195/600: open fd = 3 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.549 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:196/600: open fd = 3 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:197/600: open fd = 3 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:198/600: open fd = 3 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:199/600: open fd = 3 2024-04-25T21:17:02.550 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:200/600: open fd = 3 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:201/600: open fd = 3 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.551 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:202/600: open fd = 3 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:203/600: open fd = 3 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:204/600: open fd = 3 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:205/600: open fd = 3 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.552 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:206/600: open fd = 3 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:207/600: open fd = 3 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:208/600: open fd = 3 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:209/600: open fd = 3 2024-04-25T21:17:02.553 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:210/600: open fd = 3 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:211/600: open fd = 3 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:212/600: open fd = 3 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.554 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:213/600: open fd = 3 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:214/600: open fd = 3 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:215/600: open fd = 3 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:216/600: open fd = 3 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.555 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:217/600: open fd = 3 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:218/600: open fd = 3 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:219/600: open fd = 3 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.556 INFO:tasks.workunit.client.0.smithi037.stdout:220/600: open fd = 3 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:221/600: open fd = 3 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:222/600: open fd = 3 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:223/600: open fd = 3 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.557 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:224/600: open fd = 3 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:225/600: open fd = 3 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:226/600: open fd = 3 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:227/600: open fd = 3 2024-04-25T21:17:02.558 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:228/600: open fd = 3 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:229/600: open fd = 3 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:230/600: open fd = 3 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.559 INFO:tasks.workunit.client.0.smithi037.stdout:231/600: open fd = 3 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:232/600: open fd = 3 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:233/600: open fd = 3 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:234/600: open fd = 3 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.560 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:235/600: open fd = 3 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:236/600: open fd = 3 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:237/600: open fd = 3 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.561 INFO:tasks.workunit.client.0.smithi037.stdout:238/600: open fd = 3 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:239/600: open fd = 3 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:240/600: open fd = 3 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:241/600: open fd = 3 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.562 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:242/600: open fd = 3 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:243/600: open fd = 3 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:244/600: open fd = 3 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:245/600: open fd = 3 2024-04-25T21:17:02.563 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:246/600: open fd = 3 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:247/600: open fd = 3 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:248/600: open fd = 3 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.564 INFO:tasks.workunit.client.0.smithi037.stdout:249/600: open fd = 3 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:250/600: open fd = 3 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:02.565 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:17:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/2104699834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103b20 msgr2=0x7fe4a810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/2104699834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103b20 0x7fe4a810c000 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe490009900 tx=0x7fe49002a480 comp rx=0 tx=0).stop 2024-04-25T21:17:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/2104699834 shutdown_connections 2024-04-25T21:17:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/2104699834 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a810c5d0 0x7fe4a810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/2104699834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103b20 0x7fe4a810c000 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/2104699834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a8103200 0x7fe4a81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/2104699834 >> 172.21.15.37:0/2104699834 conn(0x7fe4a80771d0 msgr2=0x7fe4a80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/2104699834 shutdown_connections 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.473+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/2104699834 wait complete. 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 Processor -- start 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 -- start start 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a8103b20 0x7fe4a81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a810c5d0 0x7fe4a819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8112210 con 0x7fe4a8103200 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8112090 con 0x7fe4a8103b20 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4ac9ad640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a8112390 con 0x7fe4a810c5d0 2024-04-25T21:17:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37570/0 (socket says 172.21.15.37:37570) 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 -- 172.21.15.37:0/3612480472 learned_addr learned my addr 172.21.15.37:0/3612480472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a810c5d0 0x7fe4a819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a8103b20 0x7fe4a81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 -- 172.21.15.37:0/3612480472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a810c5d0 msgr2=0x7fe4a819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a810c5d0 0x7fe4a819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 -- 172.21.15.37:0/3612480472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a8103b20 msgr2=0x7fe4a81a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a8103b20 0x7fe4a81a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.474+0000 7fe4a659c640 1 -- 172.21.15.37:0/3612480472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a819afe0 con 0x7fe4a8103200 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe4a659c640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d770 tx=0x7fe49800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe49f7fe640 1 -- 172.21.15.37:0/3612480472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498017070 con 0x7fe4a8103200 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a819b2d0 con 0x7fe4a8103200 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe49f7fe640 1 -- 172.21.15.37:0/3612480472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980054d0 con 0x7fe4a8103200 2024-04-25T21:17:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe49f7fe640 1 -- 172.21.15.37:0/3612480472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498004c30 con 0x7fe4a8103200 2024-04-25T21:17:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.475+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a81a75e0 con 0x7fe4a8103200 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.477+0000 7fe49f7fe640 1 -- 172.21.15.37:0/3612480472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe498004e10 con 0x7fe4a8103200 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 Processor -- start 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 -- start start 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe49f7fe640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 0x7fe4940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed876c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 0x7fe4940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed876c800 0x558ed876ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe49f7fe640 1 -- 172.21.15.37:0/3612480472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fe4980d0330 con 0x7fe4a8103200 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558ed86c6970 con 0x558ed8761620 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558ed86634a0 con 0x558ed8761f40 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4af79e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558ed86c2930 con 0x558ed876c800 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37572/0 (socket says 172.21.15.37:37572) 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.478+0000 7fe4a659c640 1 -- 172.21.15.37:0/1980853074 learned_addr learned my addr 172.21.15.37:0/1980853074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed876c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed876c800 0x558ed876ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 -- 172.21.15.37:0/1980853074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed876c800 msgr2=0x558ed876ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed876c800 0x558ed876ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 -- 172.21.15.37:0/1980853074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 msgr2=0x558ed876c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed876c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 -- 172.21.15.37:0/1980853074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed83d9210 con 0x558ed8761620 2024-04-25T21:17:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 0x7fe4940a6280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806b860 tx=0x7fe4a0011810 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe498005c20 tx=0x7fe49809ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e00baba2bc3da40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe49cff9640 1 -- 172.21.15.37:0/1980853074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980cf050 con 0x558ed8761620 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe49cff9640 1 -- 172.21.15.37:0/1980853074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980a4ee0 con 0x558ed8761620 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.479+0000 7fe49cff9640 1 -- 172.21.15.37:0/1980853074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498020040 con 0x558ed8761620 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- 172.21.15.37:0/1980853074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 msgr2=0x558ed8761a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe498005c20 tx=0x7fe49809ca30 comp rx=0 tx=0).stop 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- 172.21.15.37:0/1980853074 shutdown_connections 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed876c800 0x558ed876ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed876c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1980853074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8761a00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- 172.21.15.37:0/1980853074 >> 172.21.15.37:0/1980853074 conn(0x558ed875db30 msgr2=0x558ed875df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- 172.21.15.37:0/1980853074 shutdown_connections 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- 172.21.15.37:0/1980853074 wait complete. 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.480+0000 7fe4af79e740 1 -- start start 2024-04-25T21:17:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 Processor -- start 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 0x558ed876c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed8769430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558ed876f3d0 con 0x558ed876c800 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558ed86610b0 con 0x558ed8761f40 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558ed876f550 con 0x558ed8761620 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4af79e740 10 client.0 _ll_register_callbacks cb 0x558ed864ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 0x558ed876c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37580/0 (socket says 172.21.15.37:37580) 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a6d9d640 1 -- 172.21.15.37:0/1480864678 learned_addr learned my addr 172.21.15.37:0/1480864678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed8769430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 -- 172.21.15.37:0/1480864678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 msgr2=0x558ed8769430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed8769430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 -- 172.21.15.37:0/1480864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 msgr2=0x558ed8769970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a659c640 1 -- 172.21.15.37:0/1480864678 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed84a5620 con 0x558ed8761620 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.481+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4a659c640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 0x558ed876c140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x558ed87624b0 tx=0x7fe498098b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae09bf954bdda16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980b9020 con 0x558ed8761620 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4980a4ee0 con 0x558ed8761620 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fe4980a4c50 con 0x558ed8761620 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4980cf030 con 0x558ed8761620 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 10 client.25870 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ed876f3d0 con 0x558ed8761620 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980a4ee0 con 0x558ed8761620 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4980981c0 con 0x558ed8761620 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 10 client.25870 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 20 client.25870 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed877c460 0x558ed877e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 4 client.25870 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.482+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ed86d4390 con 0x558ed877c460 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.483+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed877c460 0x558ed877e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.483+0000 7fe48dffb640 10 client.25870 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.483+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed877c460 0x558ed877e850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f359ce914cadf650 server_cookie=92fae9dde236967b in_seq=0 out_seq=0 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe48dffb640 1 -- 172.21.15.37:0/1480864678 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ed86d4390 con 0x558ed877c460 2024-04-25T21:17:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe48dffb640 10 client.25870 handle_command_reply: tid=0 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 client.25870 shutdown 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 msgr2=0x558ed876c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 0x558ed876c140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x558ed87624b0 tx=0x7fe498098b60 comp rx=0 tx=0).stop 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 shutdown_connections 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed877c460 0x558ed877e850 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed8769970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761f40 0x558ed8769430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe48dffb640 0 client.25870 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 --2- 172.21.15.37:0/1480864678 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761620 0x558ed876c140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 >> 172.21.15.37:0/1480864678 conn(0x558ed875db30 msgr2=0x558ed87638b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 shutdown_connections 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 1 -- 172.21.15.37:0/1480864678 wait complete. 2024-04-25T21:17:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.484+0000 7fe4af79e740 20 client.25870 trim_cache size 0 max 16384 2024-04-25T21:17:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 Processor -- start 2024-04-25T21:17:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 -- start start 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8764c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed876c800 0x558ed876bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81afd70 con 0x558ed8761620 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81b18b0 con 0x558ed876c800 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.499+0000 7fe4af79e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81b0440 con 0x558ed8761f40 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37596/0 (socket says 172.21.15.37:37596) 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 -- 172.21.15.37:0/4038596766 learned_addr learned my addr 172.21.15.37:0/4038596766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8764c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 -- 172.21.15.37:0/4038596766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 msgr2=0x558ed8764c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8764c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 -- 172.21.15.37:0/4038596766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed876c800 msgr2=0x558ed876bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed876c800 0x558ed876bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed876c800 0x558ed876bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 -- 172.21.15.37:0/4038596766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ed876ebf0 con 0x558ed8761620 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe4a659c640 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe498009610 tx=0x7fe498098640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27adabbf5753934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe48ffff640 1 -- 172.21.15.37:0/4038596766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498014020 con 0x558ed8761620 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe48ffff640 1 -- 172.21.15.37:0/4038596766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980a4ee0 con 0x558ed8761620 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.500+0000 7fe48ffff640 1 -- 172.21.15.37:0/4038596766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4980b9070 con 0x558ed8761620 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- 172.21.15.37:0/4038596766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 msgr2=0x558ed8778290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe498009610 tx=0x7fe498098640 comp rx=0 tx=0).stop 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- 172.21.15.37:0/4038596766 shutdown_connections 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed876c800 0x558ed876bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8764c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- 172.21.15.37:0/4038596766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed8761620 0x558ed8778290 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- 172.21.15.37:0/4038596766 >> 172.21.15.37:0/4038596766 conn(0x558ed875db30 msgr2=0x558ed8763860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- 172.21.15.37:0/4038596766 shutdown_connections 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- 172.21.15.37:0/4038596766 wait complete. 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- start start 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 Processor -- start 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761620 0x558ed87747a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8774ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81b11e0 con 0x558ed876c800 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81b0b10 con 0x558ed8761620 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a81add70 con 0x558ed8761f40 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.501+0000 7fe4af79e740 10 client.0 _ll_register_callbacks cb 0x7fe498009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37600/0 (socket says 172.21.15.37:37600) 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 -- 172.21.15.37:0/644270778 learned_addr learned my addr 172.21.15.37:0/644270778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a5d9b640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8774ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a659c640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761620 0x558ed87747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 -- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 msgr2=0x558ed8774ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8774ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 -- 172.21.15.37:0/644270778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761620 msgr2=0x558ed87747a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761620 0x558ed87747a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 -- 172.21.15.37:0/644270778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ed8765cf0 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4a6d9d640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009560 tx=0x7fe4a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ae679f9770ec8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0017040 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4980a4ee0 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.502+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8513+0+0 (secure 0 0 0) 0x7fe4a0020b40 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4a0020d40 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 10 client.16413 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ed87e2090 con 0x558ed876c800 2024-04-25T21:17:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4980a4ee0 con 0x558ed876c800 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4a00057a0 con 0x558ed876c800 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 10 client.16413 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 20 client.16413 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed87e2860 0x558ed87e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 4 client.16413 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ed86ce200 con 0x558ed87e2860 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.503+0000 7fe4a659c640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed87e2860 0x558ed87e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.504+0000 7fe48dffb640 10 client.16413 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.504+0000 7fe4a659c640 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed87e2860 0x558ed87e4c80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c49c72dae41e508 server_cookie=9b155abf419ea47a in_seq=0 out_seq=0 2024-04-25T21:17:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe48dffb640 1 -- 172.21.15.37:0/644270778 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ed86ce200 con 0x558ed87e2860 2024-04-25T21:17:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe48dffb640 10 client.16413 handle_command_reply: tid=0 2024-04-25T21:17:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 client.16413 shutdown 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 msgr2=0x558ed87656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009560 tx=0x7fe4a0004ab0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 shutdown_connections 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558ed87e2860 0x558ed87e4c80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558ed876c800 0x558ed87656e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe48dffb640 0 client.16413 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558ed8761f40 0x558ed8774ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 --2- 172.21.15.37:0/644270778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558ed8761620 0x558ed87747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 >> 172.21.15.37:0/644270778 conn(0x558ed875db30 msgr2=0x558ed875c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 shutdown_connections 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 1 -- 172.21.15.37:0/644270778 wait complete. 2024-04-25T21:17:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.519+0000 7fe4af79e740 20 client.16413 trim_cache size 0 max 16384 2024-04-25T21:17:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:17:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:17:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:17:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 msgr2=0x7fe4940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 0x7fe4940a6280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe4a806b860 tx=0x7fe4a0011810 comp rx=0 tx=0).stop 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 msgr2=0x7fe4a81a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d770 tx=0x7fe49800dc30 comp rx=0 tx=0).stop 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 shutdown_connections 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4940a3dc0 0x7fe4940a6280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4a810c5d0 0x7fe4a819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4a8103b20 0x7fe4a81a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 --2- 172.21.15.37:0/3612480472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4a8103200 0x7fe4a81a0660 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 >> 172.21.15.37:0/3612480472 conn(0x7fe4a80771d0 msgr2=0x7fe4a80fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 shutdown_connections 2024-04-25T21:17:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:03.520+0000 7fe4ac9ad640 1 -- 172.21.15.37:0/3612480472 wait complete. 2024-04-25T21:17:03.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:03 smithi098 ceph-mon[32569]: pgmap v264: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-25T21:17:03.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:03 smithi037 ceph-mon[29195]: pgmap v264: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-25T21:17:04.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:03 smithi177 ceph-mon[33043]: pgmap v264: 97 pgs: 97 active+clean; 5.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-25T21:17:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:05 smithi098 ceph-mon[32569]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 110 KiB/s wr, 43 op/s 2024-04-25T21:17:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:05 smithi037 ceph-mon[29195]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 110 KiB/s wr, 43 op/s 2024-04-25T21:17:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:05 smithi177 ceph-mon[33043]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 110 KiB/s wr, 43 op/s 2024-04-25T21:17:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:07 smithi098 ceph-mon[32569]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 33 op/s 2024-04-25T21:17:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:07 smithi098 ceph-mon[32569]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:17:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:07 smithi037 ceph-mon[29195]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 33 op/s 2024-04-25T21:17:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:07 smithi037 ceph-mon[29195]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:17:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:07 smithi177 ceph-mon[33043]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 86 KiB/s wr, 33 op/s 2024-04-25T21:17:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:07 smithi177 ceph-mon[33043]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:17:09.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:09 smithi098 ceph-mon[32569]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 40 op/s 2024-04-25T21:17:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:09 smithi037 ceph-mon[29195]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 40 op/s 2024-04-25T21:17:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:09 smithi177 ceph-mon[33043]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 104 KiB/s wr, 40 op/s 2024-04-25T21:17:11.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:11 smithi098 ceph-mon[32569]: pgmap v269: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:11 smithi037 ceph-mon[29195]: pgmap v269: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:11 smithi177 ceph-mon[33043]: pgmap v269: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:13.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:13 smithi098 ceph-mon[32569]: pgmap v270: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:13 smithi037 ceph-mon[29195]: pgmap v270: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:13 smithi177 ceph-mon[33043]: pgmap v270: 97 pgs: 97 active+clean; 2.0 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:17:15.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:15 smithi098 ceph-mon[32569]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:15 smithi098 ceph-mon[32569]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:17:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:15 smithi037 ceph-mon[29195]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:15.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:15 smithi037 ceph-mon[29195]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:17:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:15 smithi177 ceph-mon[33043]: pgmap v271: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:15 smithi177 ceph-mon[33043]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 -- 172.21.15.37:0/3979571891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 msgr2=0x7fb4b4104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/3979571891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b4104780 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009ca0 tx=0x7fb4a402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 -- 172.21.15.37:0/3979571891 shutdown_connections 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/3979571891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 0x7fb4b4111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/3979571891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b4104d50 0x7fb4b4108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/3979571891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b4104780 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.619+0000 7fb4ba09a640 1 -- 172.21.15.37:0/3979571891 >> 172.21.15.37:0/3979571891 conn(0x7fb4b40ffc40 msgr2=0x7fb4b4102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.620+0000 7fb4ba09a640 1 -- 172.21.15.37:0/3979571891 shutdown_connections 2024-04-25T21:17:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.620+0000 7fb4ba09a640 1 -- 172.21.15.37:0/3979571891 wait complete. 2024-04-25T21:17:16.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.620+0000 7fb4ba09a640 1 Processor -- start 2024-04-25T21:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.620+0000 7fb4ba09a640 1 -- start start 2024-04-25T21:17:16.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 0x7fb4b41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b419a7a0 0x7fb4b419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4114400 con 0x7fb4b41043a0 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4114280 con 0x7fb4b4109470 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4ba09a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4114580 con 0x7fb4b419a7a0 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53532/0 (socket says 172.21.15.37:53532) 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 -- 172.21.15.37:0/1209619180 learned_addr learned my addr 172.21.15.37:0/1209619180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 0x7fb4b41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b419a7a0 0x7fb4b419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 -- 172.21.15.37:0/1209619180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b419a7a0 msgr2=0x7fb4b419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b419a7a0 0x7fb4b419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 -- 172.21.15.37:0/1209619180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 msgr2=0x7fb4b41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 0x7fb4b41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 -- 172.21.15.37:0/1209619180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419b3a0 con 0x7fb4b41043a0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.621+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005e30 tx=0x7fb4a4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.622+0000 7fb4b0ff9640 1 -- 172.21.15.37:0/1209619180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4003b70 con 0x7fb4b41043a0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.622+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419b630 con 0x7fb4b41043a0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.622+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b41a77d0 con 0x7fb4b41043a0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.622+0000 7fb4b0ff9640 1 -- 172.21.15.37:0/1209619180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a4003d10 con 0x7fb4b41043a0 2024-04-25T21:17:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.622+0000 7fb4b0ff9640 1 -- 172.21.15.37:0/1209619180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a402fa50 con 0x7fb4b41043a0 2024-04-25T21:17:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.624+0000 7fb4b0ff9640 1 -- 172.21.15.37:0/1209619180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4a402fbf0 con 0x7fb4b41043a0 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.624+0000 7fb4bcdb1740 1 Processor -- start 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.624+0000 7fb4bcdb1740 1 -- start start 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18c620 0x55913c18ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55913c0f1970 con 0x55913c18cf40 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55913c08e4a0 con 0x55913c18c620 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55913c0ed930 con 0x55913c197800 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b0ff9640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 0x7fb49c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53542/0 (socket says 172.21.15.37:53542) 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 -- 172.21.15.37:0/2419074911 learned_addr learned my addr 172.21.15.37:0/2419074911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18c620 0x55913c18ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 0x7fb49c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b0ff9640 1 -- 172.21.15.37:0/1209619180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fb4a40f1cc0 con 0x7fb4b41043a0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 -- 172.21.15.37:0/2419074911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 msgr2=0x55913c199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 -- 172.21.15.37:0/2419074911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18c620 msgr2=0x55913c18ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18c620 0x55913c18ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b37fe640 1 -- 172.21.15.37:0/2419074911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913be04210 con 0x55913c18cf40 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.625+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40ef520 tx=0x7fb4a40bdc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8611fb4df535c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 0x7fb49c0a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005fd0 tx=0x7fb4ac005850 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb499ffb640 1 -- 172.21.15.37:0/2419074911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40bd670 con 0x55913c18cf40 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb499ffb640 1 -- 172.21.15.37:0/2419074911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a4036030 con 0x55913c18cf40 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb499ffb640 1 -- 172.21.15.37:0/2419074911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40ef630 con 0x55913c18cf40 2024-04-25T21:17:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2419074911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 msgr2=0x55913c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40ef520 tx=0x7fb4a40bdc10 comp rx=0 tx=0).stop 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2419074911 shutdown_connections 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1972c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2419074911 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18c620 0x55913c18ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.626+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2419074911 >> 172.21.15.37:0/2419074911 conn(0x55913c188b30 msgr2=0x55913c188f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2419074911 shutdown_connections 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2419074911 wait complete. 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- start start 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 Processor -- start 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18cf40 0x55913c194480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c194a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55913c08e4a0 con 0x55913c18c620 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55913c08c0b0 con 0x55913c18cf40 2024-04-25T21:17:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.627+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55913c0f1970 con 0x55913c197800 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4bcdb1740 10 client.0 _ll_register_callbacks cb 0x55913c079d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53552/0 (socket says 172.21.15.37:53552) 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 -- 172.21.15.37:0/61988591 learned_addr learned my addr 172.21.15.37:0/61988591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c194a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18cf40 0x55913c194480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 -- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 msgr2=0x55913c194a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c194a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 -- 172.21.15.37:0/61988591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18cf40 msgr2=0x55913c194480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18cf40 0x55913c194480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 -- 172.21.15.37:0/61988591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913bed0620 con 0x55913c18c620 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40bdca0 tx=0x7fb4a40ba5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8324bfe6b6d4be31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb492ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40bd670 con 0x55913c18c620 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fb4a40c6700 con 0x55913c18c620 2024-04-25T21:17:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a4036030 con 0x55913c18c620 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4a40bdd10 con 0x55913c18c620 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4bcdb1740 10 client.16425 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.628+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55913c1a5ee0 con 0x55913c18c620 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a4036030 con 0x55913c18c620 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb4a40ef5a0 con 0x55913c18c620 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 10 client.16425 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 20 client.16425 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c1a7100 0x55913c1a9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 4 client.16425 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55913c0ff390 con 0x55913c1a7100 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c1a7100 0x55913c1a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.630+0000 7fb492ffd640 10 client.16425 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.629+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c1a7100 0x55913c1a9510 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a0e6b240921b17 server_cookie=42ac10bed24f9d8d in_seq=0 out_seq=0 2024-04-25T21:17:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.630+0000 7fb492ffd640 1 -- 172.21.15.37:0/61988591 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55913c0ff390 con 0x55913c1a7100 2024-04-25T21:17:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.630+0000 7fb492ffd640 10 client.16425 handle_command_reply: tid=0 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.630+0000 7fb4bcdb1740 1 client.16425 shutdown 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 msgr2=0x55913c197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40bdca0 tx=0x7fb4a40ba5b0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 shutdown_connections 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c1a7100 0x55913c1a9510 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c197800 0x55913c194a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c18cf40 0x55913c194480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/61988591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18c620 0x55913c197190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb492ffd640 0 client.16425 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 >> 172.21.15.37:0/61988591 conn(0x55913c188b30 msgr2=0x55913c18e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 shutdown_connections 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/61988591 wait complete. 2024-04-25T21:17:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.631+0000 7fb4bcdb1740 20 client.16425 trim_cache size 0 max 16384 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 Processor -- start 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 -- start start 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c1af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a000e6c0 con 0x55913c18cf40 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0010200 con 0x55913c197800 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a000ed90 con 0x55913c18c620 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53566/0 (socket says 172.21.15.37:53566) 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/3739497472 learned_addr learned my addr 172.21.15.37:0/3739497472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:16.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c1af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/3739497472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 msgr2=0x55913c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/3739497472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 msgr2=0x55913c1af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c1af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.646+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/3739497472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55913c184500 con 0x55913c18cf40 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009c70 tx=0x7fb4a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b081caf17061008d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4997fa640 1 -- 172.21.15.37:0/3739497472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004250 con 0x55913c18cf40 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4997fa640 1 -- 172.21.15.37:0/3739497472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8026070 con 0x55913c18cf40 2024-04-25T21:17:16.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4997fa640 1 -- 172.21.15.37:0/3739497472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80038e0 con 0x55913c18cf40 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/3739497472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 msgr2=0x55913c1aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.647+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009c70 tx=0x7fb4a8005f90 comp rx=0 tx=0).stop 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/3739497472 shutdown_connections 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c1af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c1aede0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/3739497472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/3739497472 >> 172.21.15.37:0/3739497472 conn(0x55913c188b30 msgr2=0x55913c187cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/3739497472 shutdown_connections 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/3739497472 wait complete. 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- start start 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 Processor -- start 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c19d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a000fb30 con 0x55913c18cf40 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a000f460 con 0x55913c197800 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a000c6c0 con 0x55913c18c620 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.648+0000 7fb4bcdb1740 10 client.0 _ll_register_callbacks cb 0x7fb4a4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53568/0 (socket says 172.21.15.37:53568) 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/2587916923 learned_addr learned my addr 172.21.15.37:0/2587916923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b3fff640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c19d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 msgr2=0x55913c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/2587916923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 msgr2=0x55913c19d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c19d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 -- 172.21.15.37:0/2587916923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55913c19d930 con 0x55913c18cf40 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4b2ffd640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009780 tx=0x7fb4a80097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea276f51dd77b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb492ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80044e0 con 0x55913c18cf40 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fb4a8020110 con 0x55913c18cf40 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a8026070 con 0x55913c18cf40 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4a8038050 con 0x55913c18cf40 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4bcdb1740 10 client.16431 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:16.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.649+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55913c20d740 con 0x55913c18cf40 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a8026070 con 0x55913c18cf40 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb4a8003ca0 con 0x55913c18cf40 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 10 client.16431 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 20 client.16431 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c20de80 0x55913c2102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 4 client.16431 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55913c0f9200 con 0x55913c20de80 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c20de80 0x55913c2102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb4b37fe640 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c20de80 0x55913c2102a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320e095f26db091a server_cookie=7b17c1c1bd060e96 in_seq=0 out_seq=0 2024-04-25T21:17:16.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.650+0000 7fb492ffd640 10 client.16431 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb492ffd640 1 -- 172.21.15.37:0/2587916923 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55913c0f9200 con 0x55913c20de80 2024-04-25T21:17:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb492ffd640 10 client.16431 handle_command_reply: tid=0 2024-04-25T21:17:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 client.16431 shutdown 2024-04-25T21:17:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 msgr2=0x55913c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009780 tx=0x7fb4a80097b0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 shutdown_connections 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55913c20de80 0x55913c2102a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55913c197800 0x55913c19d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55913c18cf40 0x55913c19cb50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb492ffd640 0 client.16431 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 --2- 172.21.15.37:0/2587916923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55913c18c620 0x55913c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 >> 172.21.15.37:0/2587916923 conn(0x55913c188b30 msgr2=0x55913c193ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 shutdown_connections 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 1 -- 172.21.15.37:0/2587916923 wait complete. 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.670+0000 7fb4bcdb1740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T21:17:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:17:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:17:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:17:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 msgr2=0x7fb49c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 0x7fb49c0a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005fd0 tx=0x7fb4ac005850 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 msgr2=0x7fb4b41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005e30 tx=0x7fb4a4031840 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 shutdown_connections 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb49c0a3dc0 0x7fb49c0a6280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb4b419a7a0 0x7fb4b419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4b4109470 0x7fb4b41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 --2- 172.21.15.37:0/1209619180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b41043a0 0x7fb4b41a0690 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 >> 172.21.15.37:0/1209619180 conn(0x7fb4b40ffc40 msgr2=0x7fb4b4101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 shutdown_connections 2024-04-25T21:17:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:16.671+0000 7fb4ba09a640 1 -- 172.21.15.37:0/1209619180 wait complete. 2024-04-25T21:17:17.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:17 smithi098 ceph-mon[32569]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T21:17:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:17 smithi037 ceph-mon[29195]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T21:17:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:17 smithi177 ceph-mon[33043]: pgmap v273: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 16 op/s 2024-04-25T21:17:19.892 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:19 smithi177 ceph-mon[33043]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:19.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:19 smithi098 ceph-mon[32569]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:19 smithi037 ceph-mon[29195]: pgmap v274: 97 pgs: 97 active+clean; 2.2 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:21.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:21 smithi098 ceph-mon[32569]: pgmap v275: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:21 smithi037 ceph-mon[29195]: pgmap v275: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:21 smithi177 ceph-mon[33043]: pgmap v275: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:23 smithi098 ceph-mon[32569]: pgmap v276: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:23 smithi037 ceph-mon[29195]: pgmap v276: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:23 smithi177 ceph-mon[33043]: pgmap v276: 97 pgs: 97 active+clean; 2.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-25T21:17:25.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:25 smithi098 ceph-mon[32569]: pgmap v277: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:17:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:25 smithi037 ceph-mon[29195]: pgmap v277: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:17:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:25 smithi177 ceph-mon[33043]: pgmap v277: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:17:27.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:27 smithi098 ceph-mon[32569]: pgmap v278: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-25T21:17:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:27 smithi037 ceph-mon[29195]: pgmap v278: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-25T21:17:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:27 smithi177 ceph-mon[33043]: pgmap v278: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-25T21:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 -- 172.21.15.37:0/2973189051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce4410e910 msgr2=0x7fce44110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 --2- 172.21.15.37:0/2973189051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce4410e910 0x7fce44110d00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009ca0 tx=0x7fce3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.772 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 -- 172.21.15.37:0/2973189051 shutdown_connections 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 --2- 172.21.15.37:0/2973189051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce4410e910 0x7fce44110d00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 --2- 172.21.15.37:0/2973189051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44101680 0x7fce4410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 --2- 172.21.15.37:0/2973189051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce44100d60 0x7fce44101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 -- 172.21.15.37:0/2973189051 >> 172.21.15.37:0/2973189051 conn(0x7fce44077fd0 msgr2=0x7fce440783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 -- 172.21.15.37:0/2973189051 shutdown_connections 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.770+0000 7fce4be10640 1 -- 172.21.15.37:0/2973189051 wait complete. 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 Processor -- start 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 -- start start 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 0x7fce441a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4410e910 0x7fce4419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce441143b0 con 0x7fce44101680 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce44114230 con 0x7fce44100d60 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4be10640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce44114530 con 0x7fce4410e910 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce493ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce49bac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 0x7fce441a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce493ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47872/0 (socket says 172.21.15.37:47872) 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce493ab640 1 -- 172.21.15.37:0/1204274715 learned_addr learned my addr 172.21.15.37:0/1204274715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.771+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4410e910 0x7fce4419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 -- 172.21.15.37:0/1204274715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4410e910 msgr2=0x7fce4419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4410e910 0x7fce4419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 -- 172.21.15.37:0/1204274715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 msgr2=0x7fce441a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 0x7fce441a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 -- 172.21.15.37:0/1204274715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4419af90 con 0x7fce44101680 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce49bac640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 0x7fce441a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce493ab640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce3affd640 1 -- 172.21.15.37:0/1204274715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cd80 con 0x7fce44101680 2024-04-25T21:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4419b280 con 0x7fce44101680 2024-04-25T21:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce441a7540 con 0x7fce44101680 2024-04-25T21:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce3affd640 1 -- 172.21.15.37:0/1204274715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340108e0 con 0x7fce44101680 2024-04-25T21:17:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.772+0000 7fce3affd640 1 -- 172.21.15.37:0/1204274715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340188e0 con 0x7fce44101680 2024-04-25T21:17:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.774+0000 7fce3affd640 1 -- 172.21.15.37:0/1204274715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce34010430 con 0x7fce44101680 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 Processor -- start 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 -- start start 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5626ecc87970 con 0x5626ecd2d800 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5626ecc244a0 con 0x5626ecd22f40 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce3affd640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 0x7fce300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.775+0000 7fce4ec01740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626ecc83930 con 0x5626ecd22620 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce493ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce4a3ad640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:34388/0 (socket says 172.21.15.37:34388) 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 -- 172.21.15.37:0/1177703042 learned_addr learned my addr 172.21.15.37:0/1177703042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce3affd640 1 -- 172.21.15.37:0/1204274715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce340d0990 con 0x7fce44101680 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 0x7fce300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 -- 172.21.15.37:0/1177703042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 msgr2=0x5626ecd22a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd22a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 -- 172.21.15.37:0/1177703042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 msgr2=0x5626ecd2fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 -- 172.21.15.37:0/1177703042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ec99a210 con 0x5626ecd22f40 2024-04-25T21:17:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 0x7fce300a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce3c006010 tx=0x7fce3c02a8c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce49bac640 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce400115f0 tx=0x7fce40011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30703ab76e7d9487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce2ffff640 1 -- 172.21.15.37:0/1177703042 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4001e070 con 0x5626ecd22f40 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce2ffff640 1 -- 172.21.15.37:0/1177703042 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce4000faf0 con 0x5626ecd22f40 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.776+0000 7fce2ffff640 1 -- 172.21.15.37:0/1177703042 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40019b20 con 0x5626ecd22f40 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 -- 172.21.15.37:0/1177703042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 msgr2=0x5626ecd2d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce400115f0 tx=0x7fce40011ab0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 -- 172.21.15.37:0/1177703042 shutdown_connections 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd2d2c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1177703042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 -- 172.21.15.37:0/1177703042 >> 172.21.15.37:0/1177703042 conn(0x5626ecd1eb30 msgr2=0x5626ecd1ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 -- 172.21.15.37:0/1177703042 shutdown_connections 2024-04-25T21:17:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.777+0000 7fce4ec01740 1 -- 172.21.15.37:0/1177703042 wait complete. 2024-04-25T21:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 -- start start 2024-04-25T21:17:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 Processor -- start 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 0x5626ecd2d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5626ecd30430 con 0x5626ecd2d800 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5626ecc220b0 con 0x5626ecd22620 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5626ecd305b0 con 0x5626ecd22f40 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4ec01740 10 client.0 _ll_register_callbacks cb 0x5626ecc0fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:29.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4a3ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4a3ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47888/0 (socket says 172.21.15.37:47888) 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce4a3ad640 1 -- 172.21.15.37:0/2028142676 learned_addr learned my addr 172.21.15.37:0/2028142676 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce493ab640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.778+0000 7fce49bac640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 0x5626ecd2d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 -- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 msgr2=0x5626ecd2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 -- 172.21.15.37:0/2028142676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 msgr2=0x5626ecd2d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 0x5626ecd2d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 -- 172.21.15.37:0/2028142676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626eca66620 con 0x5626ecd2d800 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce49bac640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 0x5626ecd2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fce3c031970 tx=0x7fce3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd38e31a4eab0472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c051070 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce3c004240 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce4000faf0 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce3c02f350 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4ec01740 10 client.16443 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.779+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626ecd3c380 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce4000faf0 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fce3c02fb90 con 0x5626ecd2d800 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 10 client.16443 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 20 client.16443 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd3d960 0x5626ecd3fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 4 client.16443 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:17:29.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626ecc95390 con 0x5626ecd3d960 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.780+0000 7fce49bac640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd3d960 0x5626ecd3fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce49bac640 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd3d960 0x5626ecd3fde0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c29d2203a9d6f7a0 server_cookie=5dcd39112187da60 in_seq=0 out_seq=0 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce2d7fa640 10 client.16443 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce2d7fa640 1 -- 172.21.15.37:0/2028142676 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5626ecc95390 con 0x5626ecd3d960 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce2d7fa640 10 client.16443 handle_command_reply: tid=0 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce4ec01740 1 client.16443 shutdown 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 msgr2=0x5626ecd2a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.781+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fce3c031970 tx=0x7fce3c0095c0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 shutdown_connections 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd3d960 0x5626ecd3fde0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd2d800 0x5626ecd2a960 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce2d7fa640 0 client.16443 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 --2- 172.21.15.37:0/2028142676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22620 0x5626ecd2d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 >> 172.21.15.37:0/2028142676 conn(0x5626ecd1eb30 msgr2=0x5626ecd24860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 shutdown_connections 2024-04-25T21:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 1 -- 172.21.15.37:0/2028142676 wait complete. 2024-04-25T21:17:29.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.782+0000 7fce4ec01740 20 client.16443 trim_cache size 0 max 16384 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 Processor -- start 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 -- start start 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd1eb30 0x5626ecd392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce2800e760 con 0x5626ecd22620 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce280102a0 con 0x5626ecd1eb30 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4ec01740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce2800ee30 con 0x5626ecd22f40 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47904/0 (socket says 172.21.15.37:47904) 2024-04-25T21:17:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 -- 172.21.15.37:0/1337549346 learned_addr learned my addr 172.21.15.37:0/1337549346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce49bac640 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd1eb30 0x5626ecd392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 -- 172.21.15.37:0/1337549346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 msgr2=0x5626ecd456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd456b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 -- 172.21.15.37:0/1337549346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd1eb30 msgr2=0x5626ecd392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd1eb30 0x5626ecd392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.797+0000 7fce493ab640 1 -- 172.21.15.37:0/1337549346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626ecd238c0 con 0x5626ecd22620 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce493ab640 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fce340ce140 tx=0x7fce340bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b53be2a40bf4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce2f7fe640 1 -- 172.21.15.37:0/1337549346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3409c9e0 con 0x5626ecd22620 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce2f7fe640 1 -- 172.21.15.37:0/1337549346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340a0ce0 con 0x5626ecd22620 2024-04-25T21:17:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce2f7fe640 1 -- 172.21.15.37:0/1337549346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3401ece0 con 0x5626ecd22620 2024-04-25T21:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 -- 172.21.15.37:0/1337549346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 msgr2=0x5626ecd1db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fce340ce140 tx=0x7fce340bd040 comp rx=0 tx=0).stop 2024-04-25T21:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 -- 172.21.15.37:0/1337549346 shutdown_connections 2024-04-25T21:17:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22f40 0x5626ecd456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd22620 0x5626ecd1db30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 --2- 172.21.15.37:0/1337549346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd1eb30 0x5626ecd392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 -- 172.21.15.37:0/1337549346 >> 172.21.15.37:0/1337549346 conn(0x5626ecd2aac0 msgr2=0x5626ecd1d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 -- 172.21.15.37:0/1337549346 shutdown_connections 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.798+0000 7fce4ec01740 1 -- 172.21.15.37:0/1337549346 wait complete. 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 -- start start 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 Processor -- start 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd33790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd33cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce2800fbd0 con 0x5626ecd1eb30 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce2800f500 con 0x5626ecd22f40 2024-04-25T21:17:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce2800c760 con 0x5626ecd22620 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce4ec01740 10 client.0 _ll_register_callbacks cb 0x7fce3c031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce49bac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce49bac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47910/0 (socket says 172.21.15.37:47910) 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce49bac640 1 -- 172.21.15.37:0/3823553094 learned_addr learned my addr 172.21.15.37:0/3823553094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.799+0000 7fce493ab640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd33790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce4a3ad640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd33cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 -- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 msgr2=0x5626ecd33790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd33790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 -- 172.21.15.37:0/3823553094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 msgr2=0x5626ecd33cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd33cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 -- 172.21.15.37:0/3823553094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626ecda2f60 con 0x5626ecd1eb30 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce49bac640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fce40014010 tx=0x7fce40005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a448bb12b88528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4001e040 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7fce4000f740 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce340a0ce0 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce40038050 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce4ec01740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5626ecda4230 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.800+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340a0ce0 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fce40019770 con 0x5626ecd1eb30 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 20 client.16449 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd2d800 0x5626ecda6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 4 client.16449 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626ecc8f200 con 0x5626ecd2d800 2024-04-25T21:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce493ab640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd2d800 0x5626ecda6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce2d7fa640 10 client.16449 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:29.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.801+0000 7fce493ab640 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd2d800 0x5626ecda6820 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4564f796b9e2a13 server_cookie=91f8068c1d0e2b7 in_seq=0 out_seq=0 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce2d7fa640 1 -- 172.21.15.37:0/3823553094 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5626ecc8f200 con 0x5626ecd2d800 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce2d7fa640 10 client.16449 handle_command_reply: tid=0 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 client.16449 shutdown 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 msgr2=0x5626ecd33250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fce40014010 tx=0x7fce40005f10 comp rx=0 tx=0).stop 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 shutdown_connections 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5626ecd2d800 0x5626ecda6820 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5626ecd22f40 0x5626ecd33cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5626ecd22620 0x5626ecd33790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce2d7fa640 0 client.16449 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 --2- 172.21.15.37:0/3823553094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5626ecd1eb30 0x5626ecd33250 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 >> 172.21.15.37:0/3823553094 conn(0x5626ecd2aac0 msgr2=0x5626ecd1dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 shutdown_connections 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 1 -- 172.21.15.37:0/3823553094 wait complete. 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.817+0000 7fce4ec01740 20 client.16449 trim_cache size 0 max 16384 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:17:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:17:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:17:29.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 msgr2=0x7fce300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 0x7fce300a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fce3c006010 tx=0x7fce3c02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 msgr2=0x7fce441a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400eeb0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 shutdown_connections 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fce300a3dc0 0x7fce300a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce4410e910 0x7fce4419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce44101680 0x7fce441a0b00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 --2- 172.21.15.37:0/1204274715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce44100d60 0x7fce441a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 >> 172.21.15.37:0/1204274715 conn(0x7fce44077fd0 msgr2=0x7fce4410fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 shutdown_connections 2024-04-25T21:17:29.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:29.818+0000 7fce4be10640 1 -- 172.21.15.37:0/1204274715 wait complete. 2024-04-25T21:17:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:29 smithi098 ceph-mon[32569]: pgmap v279: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:29 smithi037 ceph-mon[29195]: pgmap v279: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:29 smithi177 ceph-mon[33043]: pgmap v279: 97 pgs: 97 active+clean; 2.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:31.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:31 smithi098 ceph-mon[32569]: pgmap v280: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:17:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:31 smithi037 ceph-mon[29195]: pgmap v280: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:17:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:31 smithi177 ceph-mon[33043]: pgmap v280: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:17:33.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:33 smithi098 ceph-mon[32569]: pgmap v281: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:33 smithi037 ceph-mon[29195]: pgmap v281: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:33 smithi177 ceph-mon[33043]: pgmap v281: 97 pgs: 97 active+clean; 2.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:35.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:35 smithi037 ceph-mon[29195]: pgmap v282: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:17:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:35 smithi177 ceph-mon[33043]: pgmap v282: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:17:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:35 smithi098 ceph-mon[32569]: pgmap v282: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:17:37.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:37 smithi037 ceph-mon[29195]: pgmap v283: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:37 smithi177 ceph-mon[33043]: pgmap v283: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:38.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:37 smithi098 ceph-mon[32569]: pgmap v283: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:39 smithi037 ceph-mon[29195]: pgmap v284: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:39 smithi177 ceph-mon[33043]: pgmap v284: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:40.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:39 smithi098 ceph-mon[32569]: pgmap v284: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:17:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:41 smithi037 ceph-mon[29195]: pgmap v285: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T21:17:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:41 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:17:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:41 smithi177 ceph-mon[33043]: pgmap v285: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T21:17:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:41 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:17:42.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:41 smithi098 ceph-mon[32569]: pgmap v285: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-25T21:17:42.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:41 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 -- 172.21.15.37:0/2666538974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 msgr2=0x7efd7c10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 --2- 172.21.15.37:0/2666538974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c10dd60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efd74009ca0 tx=0x7efd7402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 -- 172.21.15.37:0/2666538974 shutdown_connections 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 --2- 172.21.15.37:0/2666538974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c10dd60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 --2- 172.21.15.37:0/2666538974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c101a70 0x7efd7c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 --2- 172.21.15.37:0/2666538974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c1010c0 0x7efd7c1014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 -- 172.21.15.37:0/2666538974 >> 172.21.15.37:0/2666538974 conn(0x7efd7c077fd0 msgr2=0x7efd7c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 -- 172.21.15.37:0/2666538974 shutdown_connections 2024-04-25T21:17:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 -- 172.21.15.37:0/2666538974 wait complete. 2024-04-25T21:17:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.913+0000 7efd80a33640 1 Processor -- start 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 -- start start 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c1010c0 0x7efd7c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c19ced0 0x7efd7c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1110e0 con 0x7efd7c106190 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c110f60 con 0x7efd7c19ced0 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd80a33640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c111260 con 0x7efd7c1010c0 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49670/0 (socket says 172.21.15.37:49670) 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 -- 172.21.15.37:0/608220604 learned_addr learned my addr 172.21.15.37:0/608220604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd7a59c640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c1010c0 0x7efd7c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c19ced0 0x7efd7c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 -- 172.21.15.37:0/608220604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c1010c0 msgr2=0x7efd7c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c1010c0 0x7efd7c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 -- 172.21.15.37:0/608220604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c19ced0 msgr2=0x7efd7c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c19ced0 0x7efd7c19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 -- 172.21.15.37:0/608220604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19d9c0 con 0x7efd7c106190 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.914+0000 7efd79d9b640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d770 tx=0x7efd7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.915+0000 7efd6b7fe640 1 -- 172.21.15.37:0/608220604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70017070 con 0x7efd7c106190 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.915+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19dcb0 con 0x7efd7c106190 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.915+0000 7efd6b7fe640 1 -- 172.21.15.37:0/608220604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd70009c30 con 0x7efd7c106190 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.915+0000 7efd6b7fe640 1 -- 172.21.15.37:0/608220604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70017070 con 0x7efd7c106190 2024-04-25T21:17:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.915+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a54a0 con 0x7efd7c106190 2024-04-25T21:17:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.917+0000 7efd6b7fe640 1 -- 172.21.15.37:0/608220604 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd7001a070 con 0x7efd7c106190 2024-04-25T21:17:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.917+0000 7efd83820740 1 Processor -- start 2024-04-25T21:17:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 -- start start 2024-04-25T21:17:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd6b7fe640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 0x7efd640a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f0620b970 con 0x561f062b1800 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f061a84a0 con 0x561f062a6620 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd83820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561f06207930 con 0x561f062a6f40 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd6b7fe640 1 -- 172.21.15.37:0/608220604 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efd700cfab0 con 0x7efd7c106190 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 0x7efd640a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49678/0 (socket says 172.21.15.37:49678) 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7ad9d640 1 -- 172.21.15.37:0/4062371496 learned_addr learned my addr 172.21.15.37:0/4062371496 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd79d9b640 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 -- 172.21.15.37:0/4062371496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 msgr2=0x561f062b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 -- 172.21.15.37:0/4062371496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 msgr2=0x561f062b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.918+0000 7efd7a59c640 1 -- 172.21.15.37:0/4062371496 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f05f1e210 con 0x561f062a6620 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062a6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005fd0 tx=0x7efd6c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a8bedb2873f15e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 0x7efd640a6480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x561f062a3f50 tx=0x7efd7402a8c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd68ff9640 1 -- 172.21.15.37:0/4062371496 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c01c480 con 0x561f062a6620 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd68ff9640 1 -- 172.21.15.37:0/4062371496 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6c01cad0 con 0x561f062a6620 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.919+0000 7efd68ff9640 1 -- 172.21.15.37:0/4062371496 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c011bc0 con 0x561f062a6620 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- 172.21.15.37:0/4062371496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 msgr2=0x561f062a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062a6a00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efd6c005fd0 tx=0x7efd6c0107a0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- 172.21.15.37:0/4062371496 shutdown_connections 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- 172.21.15.37:0/4062371496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062a6a00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- 172.21.15.37:0/4062371496 >> 172.21.15.37:0/4062371496 conn(0x561f062a2b30 msgr2=0x561f062a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- 172.21.15.37:0/4062371496 shutdown_connections 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- 172.21.15.37:0/4062371496 wait complete. 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 -- start start 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 Processor -- start 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.920+0000 7efd83820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f062b4430 con 0x561f062b1800 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561f061a60b0 con 0x561f062a6620 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561f062b45b0 con 0x561f062a6f40 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 10 client.0 _ll_register_callbacks cb 0x561f06193d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49688/0 (socket says 172.21.15.37:49688) 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:53408/0 (socket says 172.21.15.37:53408) 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 -- 172.21.15.37:0/142070138 learned_addr learned my addr 172.21.15.37:0/142070138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd79d9b640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 -- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 msgr2=0x561f062ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 -- 172.21.15.37:0/142070138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 msgr2=0x561f062b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 -- 172.21.15.37:0/142070138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f05fea620 con 0x561f062b1800 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7a59c640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efd74003c20 tx=0x7efd74009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb48e8d45260bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74050030 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efd74047070 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.921+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd6c01cad0 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd74003e60 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 10 client.16461 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f062c0430 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6c01cad0 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efd7404a040 con 0x561f062b1800 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 10 client.16461 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 20 client.16461 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f062c18a0 0x561f062c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 4 client.16461 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f06219390 con 0x561f062c18a0 2024-04-25T21:17:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.922+0000 7efd7a59c640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f062c18a0 0x561f062c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.923+0000 7efd7a59c640 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f062c18a0 0x561f062c3cf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6eb19a9ae2be3c server_cookie=97ee1ca6437fe304 in_seq=0 out_seq=0 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.923+0000 7efd61ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.923+0000 7efd61ffb640 1 -- 172.21.15.37:0/142070138 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f06219390 con 0x561f062c18a0 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.923+0000 7efd61ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.923+0000 7efd83820740 1 client.16461 shutdown 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 msgr2=0x561f062aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efd74003c20 tx=0x7efd74009380 comp rx=0 tx=0).stop 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 shutdown_connections 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f062c18a0 0x561f062c3cf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062b1800 0x561f062aeaf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062a6f40 0x561f062ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd61ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 --2- 172.21.15.37:0/142070138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 >> 172.21.15.37:0/142070138 conn(0x561f062a2b30 msgr2=0x561f062a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 shutdown_connections 2024-04-25T21:17:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 1 -- 172.21.15.37:0/142070138 wait complete. 2024-04-25T21:17:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.924+0000 7efd83820740 20 client.16461 trim_cache size 0 max 16384 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 Processor -- start 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 -- start start 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1ab8d0 con 0x561f062a6f40 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1ad410 con 0x561f062a6620 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd83820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1abfa0 con 0x561f062b1800 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49694/0 (socket says 172.21.15.37:49694) 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 -- 172.21.15.37:0/1234617665 learned_addr learned my addr 172.21.15.37:0/1234617665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd7a59c640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 -- 172.21.15.37:0/1234617665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 msgr2=0x561f062c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 -- 172.21.15.37:0/1234617665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 msgr2=0x561f062b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 -- 172.21.15.37:0/1234617665 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f062af0f0 con 0x561f062a6f40 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd79d9b640 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd70098420 tx=0x7efd700bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fd7f0b754c609d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.939+0000 7efd63fff640 1 -- 172.21.15.37:0/1234617665 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70021070 con 0x561f062a6f40 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd63fff640 1 -- 172.21.15.37:0/1234617665 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd700236c0 con 0x561f062a6f40 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd63fff640 1 -- 172.21.15.37:0/1234617665 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700cd480 con 0x561f062a6f40 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- 172.21.15.37:0/1234617665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 msgr2=0x561f062c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd70098420 tx=0x7efd700bc040 comp rx=0 tx=0).stop 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- 172.21.15.37:0/1234617665 shutdown_connections 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062c9650 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 --2- 172.21.15.37:0/1234617665 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- 172.21.15.37:0/1234617665 >> 172.21.15.37:0/1234617665 conn(0x561f062a2b30 msgr2=0x561f062ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- 172.21.15.37:0/1234617665 shutdown_connections 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- 172.21.15.37:0/1234617665 wait complete. 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.940+0000 7efd83820740 1 -- start start 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 Processor -- start 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1acd40 con 0x561f062a6f40 2024-04-25T21:17:42.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1ac670 con 0x561f062a6620 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1a98d0 con 0x561f062b1800 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd83820740 10 client.0 _ll_register_callbacks cb 0x7efd74031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49700/0 (socket says 172.21.15.37:49700) 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 -- 172.21.15.37:0/4233828636 learned_addr learned my addr 172.21.15.37:0/4233828636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd7ad9d640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 -- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 msgr2=0x561f062b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 -- 172.21.15.37:0/4233828636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 msgr2=0x561f062b9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 -- 172.21.15.37:0/4233828636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f062b7bc0 con 0x561f062a6f40 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd79d9b640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d740 tx=0x7efd7009b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185bb5bdf8831d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.941+0000 7efd61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700b4020 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7efd700980c0 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd700236c0 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efd70021040 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 10 client.16467 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561f06327dd0 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd700236c0 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7efd70024060 con 0x561f062a6f40 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 10 client.16467 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 20 client.16467 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f06328480 0x561f0632a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 4 client.16467 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f06213200 con 0x561f06328480 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.942+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f06328480 0x561f0632a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.943+0000 7efd7a59c640 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f06328480 0x561f0632a870 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70e345ceeea26f7 server_cookie=b46b1d2b76e67754 in_seq=0 out_seq=0 2024-04-25T21:17:42.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.943+0000 7efd61ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.955+0000 7efd61ffb640 1 -- 172.21.15.37:0/4233828636 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f06213200 con 0x561f06328480 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.955+0000 7efd61ffb640 10 client.16467 handle_command_reply: tid=0 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.955+0000 7efd83820740 1 client.16467 shutdown 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 msgr2=0x561f062b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d740 tx=0x7efd7009b8a0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 shutdown_connections 2024-04-25T21:17:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561f06328480 0x561f0632a870 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561f062b1800 0x561f062b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd61ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f062a6f40 0x561f062b6ed0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 --2- 172.21.15.37:0/4233828636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561f062a6620 0x561f062b9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 >> 172.21.15.37:0/4233828636 conn(0x561f062a2b30 msgr2=0x561f062a1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 shutdown_connections 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 1 -- 172.21.15.37:0/4233828636 wait complete. 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd83820740 20 client.16467 trim_cache size 0 max 16384 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:17:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:17:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 msgr2=0x7efd640a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 0x7efd640a6480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x561f062a3f50 tx=0x7efd7402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 msgr2=0x7efd7c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d770 tx=0x7efd7000dc30 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 shutdown_connections 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efd7c19ced0 0x7efd7c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd7c106190 0x7efd7c198b50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7efd640a3fc0 0x7efd640a6480 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 --2- 172.21.15.37:0/608220604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd7c1010c0 0x7efd7c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.956+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 >> 172.21.15.37:0/608220604 conn(0x7efd7c077fd0 msgr2=0x7efd7c104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.957+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 shutdown_connections 2024-04-25T21:17:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:42.957+0000 7efd80a33640 1 -- 172.21.15.37:0/608220604 wait complete. 2024-04-25T21:17:42.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:42 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:17:42.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:42 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:17:42.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:42 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:17:43.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:42 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:17:43.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:42 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:17:43.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:42 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:17:43.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:42 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:17:43.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:42 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:17:43.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:42 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:17:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:43 smithi037 ceph-mon[29195]: pgmap v286: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:43 smithi177 ceph-mon[33043]: pgmap v286: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:44.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:43 smithi098 ceph-mon[32569]: pgmap v286: 97 pgs: 97 active+clean; 3.3 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:17:45.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:45 smithi098 ceph-mon[32569]: pgmap v287: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:17:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:45 smithi037 ceph-mon[29195]: pgmap v287: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:17:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:45 smithi177 ceph-mon[33043]: pgmap v287: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:17:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:47 smithi037 ceph-mon[29195]: pgmap v288: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:47 smithi177 ceph-mon[33043]: pgmap v288: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:47 smithi098 ceph-mon[32569]: pgmap v288: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:49 smithi037 ceph-mon[29195]: pgmap v289: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:49 smithi177 ceph-mon[33043]: pgmap v289: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:50.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:49 smithi098 ceph-mon[32569]: pgmap v289: 97 pgs: 97 active+clean; 3.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:51 smithi037 ceph-mon[29195]: pgmap v290: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:17:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:51 smithi177 ceph-mon[33043]: pgmap v290: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:17:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:51 smithi098 ceph-mon[32569]: pgmap v290: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:17:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:53 smithi037 ceph-mon[29195]: pgmap v291: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:53 smithi177 ceph-mon[33043]: pgmap v291: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:53 smithi098 ceph-mon[32569]: pgmap v291: 97 pgs: 97 active+clean; 3.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:17:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:55 smithi037 ceph-mon[29195]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:17:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 -- 172.21.15.37:0/925698627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a101010f0 msgr2=0x7f1a101014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 --2- 172.21.15.37:0/925698627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a101010f0 0x7f1a101014d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009900 tx=0x7f1a0002a410 comp rx=0 tx=0).stop 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 -- 172.21.15.37:0/925698627 shutdown_connections 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 --2- 172.21.15.37:0/925698627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a10106170 0x7f1a1010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 --2- 172.21.15.37:0/925698627 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a10101aa0 0x7f1a10105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 --2- 172.21.15.37:0/925698627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a101010f0 0x7f1a101014d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 -- 172.21.15.37:0/925698627 >> 172.21.15.37:0/925698627 conn(0x7f1a10077fd0 msgr2=0x7f1a100783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 -- 172.21.15.37:0/925698627 shutdown_connections 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.056+0000 7f1a15d84640 1 -- 172.21.15.37:0/925698627 wait complete. 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 Processor -- start 2024-04-25T21:17:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 -- start start 2024-04-25T21:17:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a101010f0 0x7f1a10198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a1019cf50 0x7f1a1019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101110b0 con 0x7f1a10106170 2024-04-25T21:17:56.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10110f30 con 0x7f1a101010f0 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a15d84640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10111230 con 0x7f1a1019cf50 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59120/0 (socket says 172.21.15.37:59120) 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a0effd640 1 -- 172.21.15.37:0/1065920471 learned_addr learned my addr 172.21.15.37:0/1065920471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a1019cf50 0x7f1a1019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.057+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a101010f0 0x7f1a10198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 -- 172.21.15.37:0/1065920471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a1019cf50 msgr2=0x7f1a1019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a1019cf50 0x7f1a1019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 -- 172.21.15.37:0/1065920471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a101010f0 msgr2=0x7f1a10198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a101010f0 0x7f1a10198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 -- 172.21.15.37:0/1065920471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1019da40 con 0x7f1a10106170 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0effd640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400d770 tx=0x7f1a0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0cff9640 1 -- 172.21.15.37:0/1065920471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04017070 con 0x7f1a10106170 2024-04-25T21:17:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0cff9640 1 -- 172.21.15.37:0/1065920471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a04009c30 con 0x7f1a10106170 2024-04-25T21:17:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1019dd30 con 0x7f1a10106170 2024-04-25T21:17:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a0cff9640 1 -- 172.21.15.37:0/1065920471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400f620 con 0x7f1a10106170 2024-04-25T21:17:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.058+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101a5520 con 0x7f1a10106170 2024-04-25T21:17:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.061+0000 7f1a0cff9640 1 -- 172.21.15.37:0/1065920471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a0401a070 con 0x7f1a10106170 2024-04-25T21:17:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.061+0000 7f1a18b78740 1 Processor -- start 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.061+0000 7f1a18b78740 1 -- start start 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x5636174262c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617428bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563617380970 con 0x56361741b620 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56361731d4a0 con 0x56361741bf40 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a18b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56361737c930 con 0x563617426800 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59122/0 (socket says 172.21.15.37:59122) 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 -- 172.21.15.37:0/191322192 learned_addr learned my addr 172.21.15.37:0/191322192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x5636174262c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0cff9640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 0x7f19f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0effd640 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617428bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 -- 172.21.15.37:0/191322192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 msgr2=0x563617428bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617428bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 -- 172.21.15.37:0/191322192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 msgr2=0x5636174262c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x5636174262c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 -- 172.21.15.37:0/191322192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563617093210 con 0x56361741b620 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 0x7f19f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0cff9640 1 -- 172.21.15.37:0/1065920471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1a040d02c0 con 0x7f1a10106170 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.062+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1a08010440 tx=0x7f1a08010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6fd8871225c5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f19f5ffb640 1 -- 172.21.15.37:0/191322192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0801d400 con 0x56361741b620 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f19f5ffb640 1 -- 172.21.15.37:0/191322192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0801da50 con 0x56361741b620 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f19f5ffb640 1 -- 172.21.15.37:0/191322192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0801c760 con 0x56361741b620 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 -- 172.21.15.37:0/191322192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 msgr2=0x56361741ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 0x7f19f80a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1a08021000 tx=0x7f1a0801e000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1a08010440 tx=0x7f1a08010900 comp rx=0 tx=0).stop 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 -- 172.21.15.37:0/191322192 shutdown_connections 2024-04-25T21:17:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617428bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x5636174262c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 --2- 172.21.15.37:0/191322192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x56361741ba00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 -- 172.21.15.37:0/191322192 >> 172.21.15.37:0/191322192 conn(0x563617417b30 msgr2=0x563617417f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.063+0000 7f1a18b78740 1 -- 172.21.15.37:0/191322192 shutdown_connections 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 -- 172.21.15.37:0/191322192 wait complete. 2024-04-25T21:17:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 -- start start 2024-04-25T21:17:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 Processor -- start 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x563617423420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617423960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563617429430 con 0x56361741b620 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56361731b0b0 con 0x56361741bf40 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5636174295b0 con 0x563617426800 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a18b78740 10 client.0 _ll_register_callbacks cb 0x563617308d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.064+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59126/0 (socket says 172.21.15.37:59126) 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/2138009125 learned_addr learned my addr 172.21.15.37:0/2138009125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617423960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0effd640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x563617423420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 msgr2=0x563617423960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617423960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/2138009125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 msgr2=0x563617423420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x563617423420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/2138009125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361715f620 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009900 tx=0x7f1a000316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b10ff30e70b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f19eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00035040 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1a00004040 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a0801da50 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a00004240 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.065+0000 7f1a18b78740 10 client.16479 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563617435390 con 0x56361741b620 2024-04-25T21:17:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0801da50 con 0x56361741b620 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1a00004850 con 0x56361741b620 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 10 client.16479 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 20 client.16479 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563617436800 0x563617438bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 4 client.16479 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56361738e390 con 0x563617436800 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.066+0000 7f1a0effd640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563617436800 0x563617438bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.067+0000 7f1a0effd640 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563617436800 0x563617438bf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc34faac23c782 server_cookie=5baf61eabfcef393 in_seq=0 out_seq=0 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.067+0000 7f19eeffd640 10 client.16479 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.067+0000 7f19eeffd640 1 -- 172.21.15.37:0/2138009125 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56361738e390 con 0x563617436800 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.067+0000 7f19eeffd640 10 client.16479 handle_command_reply: tid=0 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.067+0000 7f1a18b78740 1 client.16479 shutdown 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 msgr2=0x563617426130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009900 tx=0x7f1a000316d0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 shutdown_connections 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563617436800 0x563617438bf0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563617426800 0x563617423960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56361741bf40 0x563617423420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f19eeffd640 0 client.16479 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 --2- 172.21.15.37:0/2138009125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x563617426130 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 >> 172.21.15.37:0/2138009125 conn(0x563617417b30 msgr2=0x56361741d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 shutdown_connections 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 1 -- 172.21.15.37:0/2138009125 wait complete. 2024-04-25T21:17:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.068+0000 7f1a18b78740 20 client.16479 trim_cache size 0 max 16384 2024-04-25T21:17:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 Processor -- start 2024-04-25T21:17:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 -- start start 2024-04-25T21:17:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741bf40 0x563617425c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x56361743e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ada60 con 0x56361741b620 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101af5a0 con 0x563617426800 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a18b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ae130 con 0x56361741bf40 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59136/0 (socket says 172.21.15.37:59136) 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/371668351 learned_addr learned my addr 172.21.15.37:0/371668351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x56361743e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/371668351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741bf40 msgr2=0x563617425c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741bf40 0x563617425c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/371668351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 msgr2=0x56361743e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x56361743e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.083+0000 7f1a0f7fe640 1 -- 172.21.15.37:0/371668351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563617438f60 con 0x56361741b620 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000c00 tx=0x7f1a000037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc281967889ab052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f19f57fa640 1 -- 172.21.15.37:0/371668351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00035030 con 0x56361741b620 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f19f57fa640 1 -- 172.21.15.37:0/371668351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00031740 con 0x56361741b620 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f19f57fa640 1 -- 172.21.15.37:0/371668351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00003f70 con 0x56361741b620 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 -- 172.21.15.37:0/371668351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 msgr2=0x5636174322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000c00 tx=0x7f1a000037a0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 -- 172.21.15.37:0/371668351 shutdown_connections 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x56361743e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741bf40 0x563617425c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 --2- 172.21.15.37:0/371668351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741b620 0x5636174322f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 -- 172.21.15.37:0/371668351 >> 172.21.15.37:0/371668351 conn(0x563617417b30 msgr2=0x563617416cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 -- 172.21.15.37:0/371668351 shutdown_connections 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.084+0000 7f1a18b78740 1 -- 172.21.15.37:0/371668351 wait complete. 2024-04-25T21:17:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 -- start start 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 Processor -- start 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741b620 0x563617420310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x563617420d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101aeed0 con 0x56361741bf40 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101ae800 con 0x563617426800 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101aba60 con 0x56361741b620 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a18b78740 10 client.0 _ll_register_callbacks cb 0x7f1a00032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:17:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.085+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0f7fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741b620 0x563617420310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59144/0 (socket says 172.21.15.37:59144) 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 -- 172.21.15.37:0/658426657 learned_addr learned my addr 172.21.15.37:0/658426657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0ffff640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x563617420d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 -- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741b620 msgr2=0x563617420310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741b620 0x563617420310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 -- 172.21.15.37:0/658426657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 msgr2=0x563617420d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x563617420d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 -- 172.21.15.37:0/658426657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56361749bf90 con 0x56361741bf40 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a0effd640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400d740 tx=0x7f1a0409c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eff76a20fa7686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f19eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04021040 con 0x56361741bf40 2024-04-25T21:17:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f1a04098980 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a00031740 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1a0409c310 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a18b78740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56361749d2c0 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.086+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00031740 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1a04080080 con 0x56361741bf40 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 20 client.16485 populate_metadata read hostname 'smithi037' 2024-04-25T21:17:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56361749d990 0x56361749fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:17:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 4 client.16485 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:17:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563617388200 con 0x56361749d990 2024-04-25T21:17:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56361749d990 0x56361749fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:17:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f1a0f7fe640 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56361749d990 0x56361749fdb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c838f622066de9b2 server_cookie=332757e2eb594863 in_seq=0 out_seq=0 2024-04-25T21:17:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.087+0000 7f19eeffd640 10 client.16485 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:56.092 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:55 smithi177 ceph-mon[33043]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:17:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f19eeffd640 1 -- 172.21.15.37:0/658426657 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563617388200 con 0x56361749d990 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f19eeffd640 10 client.16485 handle_command_reply: tid=0 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 client.16485 shutdown 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 msgr2=0x563617420850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400d740 tx=0x7f1a0409c200 comp rx=0 tx=0).stop 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 shutdown_connections 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56361749d990 0x56361749fdb0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563617426800 0x563617420d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56361741bf40 0x563617420850 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 --2- 172.21.15.37:0/658426657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56361741b620 0x563617420310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f19eeffd640 0 client.16485 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 >> 172.21.15.37:0/658426657 conn(0x563617417b30 msgr2=0x563617422c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 shutdown_connections 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.104+0000 7f1a18b78740 1 -- 172.21.15.37:0/658426657 wait complete. 2024-04-25T21:17:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a18b78740 20 client.16485 trim_cache size 0 max 16384 2024-04-25T21:17:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:17:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:17:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:17:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 msgr2=0x7f19f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 0x7f19f80a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1a08021000 tx=0x7f1a0801e000 comp rx=0 tx=0).stop 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 msgr2=0x7f1a10198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400d770 tx=0x7f1a0400dc30 comp rx=0 tx=0).stop 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 shutdown_connections 2024-04-25T21:17:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f19f80a3dc0 0x7f19f80a6280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a1019cf50 0x7f1a1019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a10106170 0x7f1a10198bd0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 --2- 172.21.15.37:0/1065920471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a101010f0 0x7f1a10198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 >> 172.21.15.37:0/1065920471 conn(0x7f1a10077fd0 msgr2=0x7f1a10104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 shutdown_connections 2024-04-25T21:17:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:17:56.105+0000 7f1a15d84640 1 -- 172.21.15.37:0/1065920471 wait complete. 2024-04-25T21:17:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:55 smithi098 ceph-mon[32569]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:17:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:57 smithi037 ceph-mon[29195]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:57 smithi177 ceph-mon[33043]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:58.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:57 smithi098 ceph-mon[32569]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:17:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:17:59 smithi037 ceph-mon[29195]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:18:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:17:59 smithi177 ceph-mon[33043]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:18:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:17:59 smithi098 ceph-mon[32569]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:18:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:01 smithi037 ceph-mon[29195]: pgmap v295: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:18:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:01 smithi177 ceph-mon[33043]: pgmap v295: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:18:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:01 smithi098 ceph-mon[32569]: pgmap v295: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:18:02.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:02 smithi037 ceph-mon[29195]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:18:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:02 smithi177 ceph-mon[33043]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:18:03.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:02 smithi098 ceph-mon[32569]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:18:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:03 smithi037 ceph-mon[29195]: pgmap v297: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:18:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:03 smithi177 ceph-mon[33043]: pgmap v297: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:18:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:03 smithi098 ceph-mon[32569]: pgmap v297: 97 pgs: 97 active+clean; 4.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:18:05.062 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:251/600: open fd = 3 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:252/600: open fd = 3 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:253/600: open fd = 3 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.063 INFO:tasks.workunit.client.0.smithi037.stdout:254/600: open fd = 3 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:255/600: open fd = 3 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:256/600: open fd = 3 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:257/600: open fd = 3 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.064 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:258/600: open fd = 3 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:259/600: open fd = 3 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:260/600: open fd = 3 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:261/600: open fd = 3 2024-04-25T21:18:05.065 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:262/600: open fd = 3 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:263/600: open fd = 3 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:264/600: open fd = 3 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.066 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:265/600: open fd = 3 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:266/600: open fd = 3 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:267/600: open fd = 3 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:268/600: open fd = 3 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.067 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:269/600: open fd = 3 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:270/600: open fd = 3 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.068 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:271/600: open fd = 3 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:272/600: open fd = 3 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:273/600: open fd = 3 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.069 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:274/600: open fd = 3 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:275/600: open fd = 3 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:276/600: open fd = 3 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:277/600: open fd = 3 2024-04-25T21:18:05.070 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:278/600: open fd = 3 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:279/600: open fd = 3 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.071 INFO:tasks.workunit.client.0.smithi037.stdout:280/600: open fd = 3 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:281/600: open fd = 3 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:282/600: open fd = 3 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:283/600: open fd = 3 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.072 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:284/600: open fd = 3 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:285/600: open fd = 3 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:286/600: open fd = 3 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:287/600: open fd = 3 2024-04-25T21:18:05.073 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:288/600: open fd = 3 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:289/600: open fd = 3 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:290/600: open fd = 3 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.074 INFO:tasks.workunit.client.0.smithi037.stdout:291/600: open fd = 3 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:292/600: open fd = 3 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:293/600: open fd = 3 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:294/600: open fd = 3 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.075 INFO:tasks.workunit.client.0.smithi037.stdout:295/600: open fd = 3 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:296/600: open fd = 3 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:297/600: open fd = 3 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:298/600: open fd = 3 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:299/600: open fd = 3 2024-04-25T21:18:05.076 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:300/600: open fd = 3 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:301/600: open fd = 3 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:302/600: open fd = 3 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:303/600: open fd = 3 2024-04-25T21:18:05.077 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:304/600: open fd = 3 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:305/600: open fd = 3 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:306/600: open fd = 3 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.078 INFO:tasks.workunit.client.0.smithi037.stdout:307/600: open fd = 3 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:308/600: open fd = 3 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:309/600: open fd = 3 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:310/600: open fd = 3 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:311/600: open fd = 3 2024-04-25T21:18:05.079 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.080 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.080 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:18:05.080 INFO:tasks.workunit.client.0.smithi037.stdout:312/600: open fd = 3 2024-04-25T21:18:05.080 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.080 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:18:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:05 smithi037 ceph-mon[29195]: pgmap v298: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T21:18:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:05 smithi037 ceph-mon[29195]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:18:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:05 smithi177 ceph-mon[33043]: pgmap v298: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T21:18:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:05 smithi177 ceph-mon[33043]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:18:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:05 smithi098 ceph-mon[32569]: pgmap v298: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-25T21:18:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:05 smithi098 ceph-mon[32569]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:18:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:07 smithi037 ceph-mon[29195]: pgmap v300: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T21:18:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:07 smithi177 ceph-mon[33043]: pgmap v300: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T21:18:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:07 smithi098 ceph-mon[32569]: pgmap v300: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 129 KiB/s wr, 48 op/s 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.205+0000 7f6178423640 1 -- 172.21.15.37:0/2624306438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 msgr2=0x7f61701025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.205+0000 7f6178423640 1 --2- 172.21.15.37:0/2624306438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701025a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02a410 comp rx=0 tx=0).stop 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 -- 172.21.15.37:0/2624306438 shutdown_connections 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 --2- 172.21.15.37:0/2624306438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6170107290 0x7f617010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 --2- 172.21.15.37:0/2624306438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170102b70 0x7f6170106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 --2- 172.21.15.37:0/2624306438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701025a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 -- 172.21.15.37:0/2624306438 >> 172.21.15.37:0/2624306438 conn(0x7f61700fda20 msgr2=0x7f61700ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 -- 172.21.15.37:0/2624306438 shutdown_connections 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 -- 172.21.15.37:0/2624306438 wait complete. 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 Processor -- start 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.206+0000 7f6178423640 1 -- start start 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 0x7f61701a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f617019a7b0 0x7f617019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61701121f0 con 0x7f61701021c0 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112070 con 0x7f617019a7b0 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f6178423640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6170112370 con 0x7f6170107290 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46324/0 (socket says 172.21.15.37:46324) 2024-04-25T21:18:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 -- 172.21.15.37:0/506356316 learned_addr learned my addr 172.21.15.37:0/506356316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61769c0640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f617019a7b0 0x7f617019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61759be640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 0x7f61701a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 -- 172.21.15.37:0/506356316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 msgr2=0x7f61701a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 0x7f61701a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 -- 172.21.15.37:0/506356316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f617019a7b0 msgr2=0x7f617019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f617019a7b0 0x7f617019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 -- 172.21.15.37:0/506356316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617019b340 con 0x7f61701021c0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61759be640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 0x7f61701a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61761bf640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f616c005940 tx=0x7f616c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.207+0000 7f61677fe640 1 -- 172.21.15.37:0/506356316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c03f030 con 0x7f61701021c0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.208+0000 7f61677fe640 1 -- 172.21.15.37:0/506356316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616c003cb0 con 0x7f61701021c0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.208+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617019b5d0 con 0x7f61701021c0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.208+0000 7f61677fe640 1 -- 172.21.15.37:0/506356316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c035070 con 0x7f61701021c0 2024-04-25T21:18:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.208+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61701a77b0 con 0x7f61701021c0 2024-04-25T21:18:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.210+0000 7f61677fe640 1 -- 172.21.15.37:0/506356316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f616c042030 con 0x7f61701021c0 2024-04-25T21:18:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.210+0000 7f617b214740 1 Processor -- start 2024-04-25T21:18:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 -- start start 2024-04-25T21:18:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13c1800 0x5590b13c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590b131b970 con 0x5590b13b6620 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590b12b84a0 con 0x5590b13b6f40 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f617b214740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5590b1317930 con 0x5590b13c1800 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61677fe640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 0x7f615c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46332/0 (socket says 172.21.15.37:46332) 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61769c0640 1 -- 172.21.15.37:0/2143951726 learned_addr learned my addr 172.21.15.37:0/2143951726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61769c0640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 0x7f615c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61759be640 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13c1800 0x5590b13c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61677fe640 1 -- 172.21.15.37:0/506356316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f616c0f10e0 con 0x7f61701021c0 2024-04-25T21:18:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 -- 172.21.15.37:0/2143951726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13c1800 msgr2=0x5590b13c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13c1800 0x5590b13c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 -- 172.21.15.37:0/2143951726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 msgr2=0x5590b13b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.211+0000 7f61761bf640 1 -- 172.21.15.37:0/2143951726 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b102e210 con 0x5590b13b6f40 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.212+0000 7f61769c0640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 0x7f615c0a6300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f61680098b0 tx=0x7f6168009b50 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.212+0000 7f61761bf640 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13c12c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ee7d0 tx=0x7f616c0b24b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67a801aeefaa7c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.212+0000 7f6164bfe640 1 -- 172.21.15.37:0/2143951726 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c049040 con 0x5590b13b6f40 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.212+0000 7f6164bfe640 1 -- 172.21.15.37:0/2143951726 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616c0bd720 con 0x5590b13b6f40 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.212+0000 7f6164bfe640 1 -- 172.21.15.37:0/2143951726 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0b9690 con 0x5590b13b6f40 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- 172.21.15.37:0/2143951726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 msgr2=0x5590b13c12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13c12c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f616c0ee7d0 tx=0x7f616c0b24b0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- 172.21.15.37:0/2143951726 shutdown_connections 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13c1800 0x5590b13c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13c12c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 --2- 172.21.15.37:0/2143951726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- 172.21.15.37:0/2143951726 >> 172.21.15.37:0/2143951726 conn(0x5590b13b2b30 msgr2=0x5590b13b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- 172.21.15.37:0/2143951726 shutdown_connections 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- 172.21.15.37:0/2143951726 wait complete. 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 -- start start 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.213+0000 7f617b214740 1 Processor -- start 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5590b13c41f0 con 0x5590b13c1800 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5590b12b60b0 con 0x5590b13b6620 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5590b13c4370 con 0x5590b13b6f40 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f617b214740 10 client.0 _ll_register_callbacks cb 0x5590b12a3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:38060/0 (socket says 172.21.15.37:38060) 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 -- 172.21.15.37:0/394314414 learned_addr learned my addr 172.21.15.37:0/394314414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46346/0 (socket says 172.21.15.37:46346) 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61759be640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 -- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 msgr2=0x5590b13be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 -- 172.21.15.37:0/394314414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 msgr2=0x5590b13beac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 -- 172.21.15.37:0/394314414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b10fa620 con 0x5590b13b6620 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61769c0640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f61761bf640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5760 tx=0x7f616c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58ac56c5d2712ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f6159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:18:09.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.214+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0d6060 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f616c0b9690 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f616c0bd720 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f616c049070 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 10 client.25859 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590b13c41f0 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616c0bd720 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f616c03bbf0 con 0x5590b13b6620 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 10 client.25859 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 20 client.25859 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b13d12a0 0x5590b13d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 4 client.25859 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b1329390 con 0x5590b13d12a0 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.215+0000 7f61759be640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b13d12a0 0x5590b13d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.216+0000 7f61759be640 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b13d12a0 0x5590b13d3690 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0735a128888d42 server_cookie=4609998028542d70 in_seq=0 out_seq=0 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.216+0000 7f6159ffb640 10 client.25859 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f6159ffb640 1 -- 172.21.15.37:0/394314414 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590b1329390 con 0x5590b13d12a0 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f6159ffb640 10 client.25859 handle_command_reply: tid=0 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 client.25859 shutdown 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 msgr2=0x5590b13c11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c5760 tx=0x7f616c005c00 comp rx=0 tx=0).stop 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 shutdown_connections 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b13d12a0 0x5590b13d3690 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13beac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f6159ffb640 0 client.25859 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 --2- 172.21.15.37:0/394314414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6620 0x5590b13c11c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 >> 172.21.15.37:0/394314414 conn(0x5590b13b2b30 msgr2=0x5590b13b88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 shutdown_connections 2024-04-25T21:18:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 1 -- 172.21.15.37:0/394314414 wait complete. 2024-04-25T21:18:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.217+0000 7f617b214740 20 client.25859 trim_cache size 0 max 16384 2024-04-25T21:18:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 Processor -- start 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 -- start start 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6620 0x5590b13ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61701afde0 con 0x5590b13c1800 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b1920 con 0x5590b13b6f40 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f617b214740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b04b0 con 0x5590b13b6620 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61769c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46354/0 (socket says 172.21.15.37:46354) 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61769c0640 1 -- 172.21.15.37:0/29520454 learned_addr learned my addr 172.21.15.37:0/29520454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61761bf640 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6620 0x5590b13ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61759be640 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.232+0000 7f61769c0640 1 -- 172.21.15.37:0/29520454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6620 msgr2=0x5590b13ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f61769c0640 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6620 0x5590b13ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f61769c0640 1 -- 172.21.15.37:0/29520454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 msgr2=0x5590b13b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f61769c0640 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f61769c0640 1 -- 172.21.15.37:0/29520454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b13bf400 con 0x5590b13c1800 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f61769c0640 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6168012a10 tx=0x7f6168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a2da6f4d10e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f615bfff640 1 -- 172.21.15.37:0/29520454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168009d40 con 0x5590b13c1800 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f615bfff640 1 -- 172.21.15.37:0/29520454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6168012c60 con 0x5590b13c1800 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f615bfff640 1 -- 172.21.15.37:0/29520454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6168005570 con 0x5590b13c1800 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 -- 172.21.15.37:0/29520454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 msgr2=0x5590b13c0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6168012a10 tx=0x7f6168004a40 comp rx=0 tx=0).stop 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 -- 172.21.15.37:0/29520454 shutdown_connections 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13c1800 0x5590b13c0cf0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13b6f40 0x5590b13b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 --2- 172.21.15.37:0/29520454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6620 0x5590b13ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.233+0000 7f617b214740 1 -- 172.21.15.37:0/29520454 >> 172.21.15.37:0/29520454 conn(0x5590b13b2b30 msgr2=0x5590b13cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- 172.21.15.37:0/29520454 shutdown_connections 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- 172.21.15.37:0/29520454 wait complete. 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- start start 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 Processor -- start 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 0x5590b13c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b1250 con 0x5590b13b6620 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f61701b0b80 con 0x5590b13c1800 2024-04-25T21:18:09.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f61701adde0 con 0x5590b13b6f40 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f617b214740 10 client.0 _ll_register_callbacks cb 0x7f616c0bcdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46360/0 (socket says 172.21.15.37:46360) 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 -- 172.21.15.37:0/859870361 learned_addr learned my addr 172.21.15.37:0/859870361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61769c0640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 0x5590b13c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61759be640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 -- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 msgr2=0x5590b13c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 -- 172.21.15.37:0/859870361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 msgr2=0x5590b13c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.234+0000 7f61761bf640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 0x5590b13c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f61761bf640 1 -- 172.21.15.37:0/859870361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b13c7680 con 0x5590b13b6620 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f61769c0640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 0x5590b13c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f61761bf640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c1340 tx=0x7f616c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd788dadf55cd46e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c0d6040 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f616c03bbf0 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6168012c60 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f616c005d50 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 10 client.16503 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590b13c7680 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6168012c60 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f616c0b9690 con 0x5590b13b6620 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 10 client.16503 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 20 client.16503 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b1437970 0x5590b1439d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 4 client.16503 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:18:09.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.235+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590b13b22d0 con 0x5590b1437970 2024-04-25T21:18:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.236+0000 7f61759be640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b1437970 0x5590b1439d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.236+0000 7f61759be640 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b1437970 0x5590b1439d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d4966d4e70eef4 server_cookie=42202a9e931cb7be in_seq=0 out_seq=0 2024-04-25T21:18:09.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.236+0000 7f6159ffb640 10 client.16503 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f6159ffb640 1 -- 172.21.15.37:0/859870361 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590b13b22d0 con 0x5590b1437970 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f6159ffb640 10 client.16503 handle_command_reply: tid=0 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 client.16503 shutdown 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 msgr2=0x5590b13c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f616c0c1340 tx=0x7f616c02a410 comp rx=0 tx=0).stop 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 shutdown_connections 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5590b1437970 0x5590b1439d60 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5590b13c1800 0x5590b13c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5590b13b6f40 0x5590b13c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f6159ffb640 0 client.16503 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 --2- 172.21.15.37:0/859870361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5590b13b6620 0x5590b13c9720 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 >> 172.21.15.37:0/859870361 conn(0x5590b13b2b30 msgr2=0x5590b13bfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 shutdown_connections 2024-04-25T21:18:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 1 -- 172.21.15.37:0/859870361 wait complete. 2024-04-25T21:18:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.253+0000 7f617b214740 20 client.16503 trim_cache size 0 max 16384 2024-04-25T21:18:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:18:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:18:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:18:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 msgr2=0x7f615c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 0x7f615c0a6300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f61680098b0 tx=0x7f6168009b50 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 msgr2=0x7f61701a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f616c005940 tx=0x7f616c0042f0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 shutdown_connections 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f617019a7b0 0x7f617019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f615c0a3e40 0x7f615c0a6300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6170107290 0x7f61701a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 --2- 172.21.15.37:0/506356316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61701021c0 0x7f61701a0640 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 >> 172.21.15.37:0/506356316 conn(0x7f61700fda20 msgr2=0x7f61700ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 shutdown_connections 2024-04-25T21:18:09.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:09.254+0000 7f6178423640 1 -- 172.21.15.37:0/506356316 wait complete. 2024-04-25T21:18:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:09 smithi037 ceph-mon[29195]: pgmap v301: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:18:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:09 smithi177 ceph-mon[33043]: pgmap v301: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:18:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:09 smithi098 ceph-mon[32569]: pgmap v301: 97 pgs: 97 active+clean; 4.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:18:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:11 smithi037 ceph-mon[29195]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 42 op/s 2024-04-25T21:18:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:11 smithi177 ceph-mon[33043]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 42 op/s 2024-04-25T21:18:12.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:11 smithi098 ceph-mon[32569]: pgmap v302: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 118 KiB/s wr, 42 op/s 2024-04-25T21:18:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:13 smithi177 ceph-mon[33043]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-04-25T21:18:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:13 smithi098 ceph-mon[32569]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-04-25T21:18:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:13 smithi037 ceph-mon[29195]: pgmap v303: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 101 KiB/s wr, 36 op/s 2024-04-25T21:18:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:15 smithi177 ceph-mon[33043]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:18:16.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:15 smithi098 ceph-mon[32569]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:18:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:15 smithi037 ceph-mon[29195]: pgmap v304: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:18:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:17 smithi177 ceph-mon[33043]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T21:18:18.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:17 smithi098 ceph-mon[32569]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T21:18:18.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:17 smithi037 ceph-mon[29195]: pgmap v305: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 17 op/s 2024-04-25T21:18:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:19 smithi177 ceph-mon[33043]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:18:20.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:19 smithi098 ceph-mon[32569]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:18:20.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:19 smithi037 ceph-mon[29195]: pgmap v306: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:18:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:21 smithi177 ceph-mon[33043]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:18:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:21 smithi098 ceph-mon[32569]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:18:22.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:21 smithi037 ceph-mon[29195]: pgmap v307: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 -- 172.21.15.37:0/1905705825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c103b00 msgr2=0x7f258c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 --2- 172.21.15.37:0/1905705825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c103b00 0x7f258c10bfe0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2580009900 tx=0x7f258002a510 comp rx=0 tx=0).stop 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 -- 172.21.15.37:0/1905705825 shutdown_connections 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 --2- 172.21.15.37:0/1905705825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c10c520 0x7f258c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 --2- 172.21.15.37:0/1905705825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c103b00 0x7f258c10bfe0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 --2- 172.21.15.37:0/1905705825 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c1031e0 0x7f258c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 -- 172.21.15.37:0/1905705825 >> 172.21.15.37:0/1905705825 conn(0x7f258c077fd0 msgr2=0x7f258c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 -- 172.21.15.37:0/1905705825 shutdown_connections 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.354+0000 7f2591b83640 1 -- 172.21.15.37:0/1905705825 wait complete. 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 Processor -- start 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 -- start start 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c1031e0 0x7f258c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c103b00 0x7f258c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f258c112190 con 0x7f258c10c520 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f258c112010 con 0x7f258c103b00 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.355+0000 7f2591b83640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f258c112310 con 0x7f258c1031e0 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45848/0 (socket says 172.21.15.37:45848) 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 -- 172.21.15.37:0/1280798047 learned_addr learned my addr 172.21.15.37:0/1280798047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258affd640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c103b00 0x7f258c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c1031e0 0x7f258c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 -- 172.21.15.37:0/1280798047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c1031e0 msgr2=0x7f258c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c1031e0 0x7f258c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 -- 172.21.15.37:0/1280798047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c103b00 msgr2=0x7f258c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c103b00 0x7f258c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 -- 172.21.15.37:0/1280798047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c19d690 con 0x7f258c10c520 2024-04-25T21:18:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f258bfff640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f258400eaa0 tx=0x7f258400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f2588ff9640 1 -- 172.21.15.37:0/1280798047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584004d20 con 0x7f258c10c520 2024-04-25T21:18:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c19d980 con 0x7f258c10c520 2024-04-25T21:18:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f2588ff9640 1 -- 172.21.15.37:0/1280798047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2584005020 con 0x7f258c10c520 2024-04-25T21:18:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.356+0000 7f2588ff9640 1 -- 172.21.15.37:0/1280798047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25840116b0 con 0x7f258c10c520 2024-04-25T21:18:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.357+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c1a5210 con 0x7f258c10c520 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2588ff9640 1 -- 172.21.15.37:0/1280798047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f25840051d0 con 0x7f258c10c520 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 Processor -- start 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 -- start start 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0bca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b021970 con 0x55cb5b0c7800 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb5afbe4a0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2594970740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b01d930 con 0x55cb5b0bc620 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.359+0000 7f2588ff9640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 0x7f25740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45850/0 (socket says 172.21.15.37:45850) 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 -- 172.21.15.37:0/1509697546 learned_addr learned my addr 172.21.15.37:0/1509697546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258bfff640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 0x7f25740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0bca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f2588ff9640 1 -- 172.21.15.37:0/1280798047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f25840d0940 con 0x7f258c10c520 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 -- 172.21.15.37:0/1509697546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 msgr2=0x55cb5b0bca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0bca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 -- 172.21.15.37:0/1509697546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 msgr2=0x55cb5b0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 -- 172.21.15.37:0/1509697546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5ad34210 con 0x55cb5b0c7800 2024-04-25T21:18:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258bfff640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 0x7f25740a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25840ce140 tx=0x7f2584085040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f258affd640 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f258002aa30 tx=0x7f2580005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45bcc561c1bd90d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f2571ffb640 1 -- 172.21.15.37:0/1509697546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580035070 con 0x55cb5b0c7800 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f2571ffb640 1 -- 172.21.15.37:0/1509697546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25800035f0 con 0x55cb5b0c7800 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.360+0000 7f2571ffb640 1 -- 172.21.15.37:0/1509697546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800044e0 con 0x55cb5b0c7800 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- 172.21.15.37:0/1509697546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 msgr2=0x55cb5b0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f258002aa30 tx=0x7f2580005db0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- 172.21.15.37:0/1509697546 shutdown_connections 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c9bf0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 --2- 172.21.15.37:0/1509697546 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0bca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- 172.21.15.37:0/1509697546 >> 172.21.15.37:0/1509697546 conn(0x55cb5b0b8b30 msgr2=0x55cb5b0b8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- 172.21.15.37:0/1509697546 shutdown_connections 2024-04-25T21:18:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- 172.21.15.37:0/1509697546 wait complete. 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.361+0000 7f2594970740 1 -- start start 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 Processor -- start 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b0ca3d0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cb5afbc0b0 con 0x55cb5b0c7800 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cb5b0ca550 con 0x55cb5b0bc620 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f2594970740 10 client.0 _ll_register_callbacks cb 0x55cb5afa9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45862/0 (socket says 172.21.15.37:45862) 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 -- 172.21.15.37:0/1802837886 learned_addr learned my addr 172.21.15.37:0/1802837886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258affd640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258affd640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 -- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 msgr2=0x55cb5b0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 -- 172.21.15.37:0/1802837886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0c7800 msgr2=0x55cb5b0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258b7fe640 1 -- 172.21.15.37:0/1802837886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5ae00620 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.362+0000 7f258affd640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f258b7fe640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f257c0127d0 tx=0x7f257c012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c44d43b6498c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c01f060 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25800035f0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f257c010dc0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f257c01a430 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f2594970740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb5b0d6400 con 0x55cb5b0bcf40 2024-04-25T21:18:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25800035f0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f257c00e4d0 con 0x55cb5b0bcf40 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.363+0000 7f2594970740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f2594970740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f2594970740 20 client.16515 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b0d7910 0x55cb5b0d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f2594970740 4 client.16515 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb5b02f390 con 0x55cb5b0d7910 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f258affd640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b0d7910 0x55cb5b0d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f256affd640 10 client.16515 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.364+0000 7f258affd640 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b0d7910 0x55cb5b0d9d00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abba58547f138344 server_cookie=fe07df70f5a1405f in_seq=0 out_seq=0 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.365+0000 7f256affd640 1 -- 172.21.15.37:0/1802837886 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb5b02f390 con 0x55cb5b0d7910 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.365+0000 7f256affd640 10 client.16515 handle_command_reply: tid=0 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.365+0000 7f2594970740 1 client.16515 shutdown 2024-04-25T21:18:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.365+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 msgr2=0x55cb5b0c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.365+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f257c0127d0 tx=0x7f257c012c90 comp rx=0 tx=0).stop 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 shutdown_connections 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b0d7910 0x55cb5b0d9d00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c44b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f256affd640 0 client.16515 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 --2- 172.21.15.37:0/1802837886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0bc620 0x55cb5b0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 >> 172.21.15.37:0/1802837886 conn(0x55cb5b0b8b30 msgr2=0x55cb5b0be860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 shutdown_connections 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 1 -- 172.21.15.37:0/1802837886 wait complete. 2024-04-25T21:18:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.366+0000 7f2594970740 20 client.16515 trim_cache size 0 max 16384 2024-04-25T21:18:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 Processor -- start 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 -- start start 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1adaf0 con 0x55cb5b0bc620 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1af630 con 0x55cb5b0bcf40 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f2594970740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ae1c0 con 0x55cb5b0c7800 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45876/0 (socket says 172.21.15.37:45876) 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 -- 172.21.15.37:0/917416464 learned_addr learned my addr 172.21.15.37:0/917416464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 -- 172.21.15.37:0/917416464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 msgr2=0x55cb5b0be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 -- 172.21.15.37:0/917416464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 msgr2=0x55cb5b0c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258b7fe640 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.381+0000 7f258affd640 1 -- 172.21.15.37:0/917416464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb5b0bd8e0 con 0x55cb5b0bc620 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f258affd640 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f25800096c0 tx=0x7f2580005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7d57b1b41f46a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f25717fa640 1 -- 172.21.15.37:0/917416464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580057070 con 0x55cb5b0bc620 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f25717fa640 1 -- 172.21.15.37:0/917416464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2580052dc0 con 0x55cb5b0bc620 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f25717fa640 1 -- 172.21.15.37:0/917416464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580035040 con 0x55cb5b0bc620 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 -- 172.21.15.37:0/917416464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 msgr2=0x55cb5b0d3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f25800096c0 tx=0x7f2580005db0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 -- 172.21.15.37:0/917416464 shutdown_connections 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 --2- 172.21.15.37:0/917416464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0d3290 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 -- 172.21.15.37:0/917416464 >> 172.21.15.37:0/917416464 conn(0x55cb5b0b8b30 msgr2=0x55cb5b0bdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 -- 172.21.15.37:0/917416464 shutdown_connections 2024-04-25T21:18:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.382+0000 7f2594970740 1 -- 172.21.15.37:0/917416464 wait complete. 2024-04-25T21:18:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 -- start start 2024-04-25T21:18:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 Processor -- start 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1aef60 con 0x55cb5b0bc620 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1ae890 con 0x55cb5b0bcf40 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f258c1abaf0 con 0x55cb5b0c7800 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f2594970740 10 client.0 _ll_register_callbacks cb 0x7f257c00af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f258affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45880/0 (socket says 172.21.15.37:45880) 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.383+0000 7f258affd640 1 -- 172.21.15.37:0/144526092 learned_addr learned my addr 172.21.15.37:0/144526092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258b7fe640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 -- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 msgr2=0x55cb5b0c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 -- 172.21.15.37:0/144526092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 msgr2=0x55cb5b0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 -- 172.21.15.37:0/144526092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb5b0c1440 con 0x55cb5b0bc620 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258b7fe640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f258affd640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2580005ec0 tx=0x7f2580004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a4d662855d1819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f256affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580057040 con 0x55cb5b0bc620 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f2580005020 con 0x55cb5b0bc620 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2580052dc0 con 0x55cb5b0bc620 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2580035040 con 0x55cb5b0bc620 2024-04-25T21:18:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f2594970740 10 client.16521 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.384+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb5b13d9f0 con 0x55cb5b0bc620 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2580052dc0 con 0x55cb5b0bc620 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2580052230 con 0x55cb5b0bc620 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 10 client.16521 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 20 client.16521 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b13dbc0 0x55cb5b13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 4 client.16521 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb5b029200 con 0x55cb5b13dbc0 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.385+0000 7f258b7fe640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b13dbc0 0x55cb5b13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.386+0000 7f258b7fe640 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b13dbc0 0x55cb5b13ffb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a0cf6a96a2ee5 server_cookie=c77f756fe3f5297 in_seq=0 out_seq=0 2024-04-25T21:18:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.386+0000 7f256affd640 10 client.16521 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f256affd640 1 -- 172.21.15.37:0/144526092 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb5b029200 con 0x55cb5b13dbc0 2024-04-25T21:18:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f256affd640 10 client.16521 handle_command_reply: tid=0 2024-04-25T21:18:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 client.16521 shutdown 2024-04-25T21:18:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 msgr2=0x55cb5b0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2580005ec0 tx=0x7f2580004250 comp rx=0 tx=0).stop 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 shutdown_connections 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cb5b13dbc0 0x55cb5b13ffb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cb5b0c7800 0x55cb5b0c0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cb5b0bcf40 0x55cb5b0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f256affd640 0 client.16521 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 --2- 172.21.15.37:0/144526092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb5b0bc620 0x55cb5b0cf7a0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 >> 172.21.15.37:0/144526092 conn(0x55cb5b0b8b30 msgr2=0x55cb5b0be860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 shutdown_connections 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 1 -- 172.21.15.37:0/144526092 wait complete. 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.402+0000 7f2594970740 20 client.16521 trim_cache size 0 max 16384 2024-04-25T21:18:22.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:18:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:18:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:18:22.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 msgr2=0x7f25740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 0x7f25740a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f25840ce140 tx=0x7f2584085040 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 msgr2=0x7f258c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f258400eaa0 tx=0x7f258400eea0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 shutdown_connections 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f258c10c520 0x7f258c19cf60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f258c103b00 0x7f258c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f25740a3dc0 0x7f25740a6280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 --2- 172.21.15.37:0/1280798047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f258c1031e0 0x7f258c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 >> 172.21.15.37:0/1280798047 conn(0x7f258c077fd0 msgr2=0x7f258c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 shutdown_connections 2024-04-25T21:18:22.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:22.403+0000 7f2591b83640 1 -- 172.21.15.37:0/1280798047 wait complete. 2024-04-25T21:18:24.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:23 smithi037 ceph-mon[29195]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:23 smithi177 ceph-mon[33043]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:24.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:23 smithi098 ceph-mon[32569]: pgmap v308: 97 pgs: 97 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:24.723 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.3 2024-04-25T21:18:24.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:18:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:18:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:25 smithi177 ceph-mon[33043]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T21:18:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:25 smithi098 ceph-mon[32569]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T21:18:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:25 smithi037 ceph-mon[29195]: pgmap v309: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 24 op/s 2024-04-25T21:18:27.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 -- 172.21.15.37:0/3451244696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85901034b0 msgr2=0x7f859010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 --2- 172.21.15.37:0/3451244696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85901034b0 0x7f859010f8e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8580009990 tx=0x7f8580009ca0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 -- 172.21.15.37:0/3451244696 shutdown_connections 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 --2- 172.21.15.37:0/3451244696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85901034b0 0x7f859010f8e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 --2- 172.21.15.37:0/3451244696 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590108a80 0x7f8590108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 --2- 172.21.15.37:0/3451244696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8590102af0 0x7f8590102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 -- 172.21.15.37:0/3451244696 >> 172.21.15.37:0/3451244696 conn(0x7f8590079b20 msgr2=0x7f8590079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 -- 172.21.15.37:0/3451244696 shutdown_connections 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.017+0000 7f8597ba7700 1 -- 172.21.15.37:0/3451244696 wait complete. 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 Processor -- start 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 -- start start 2024-04-25T21:18:27.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590102af0 0x7f85901990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85901034b0 0x7f85901995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8590112c10 con 0x7f8590108a80 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8590112a90 con 0x7f85901034b0 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.018+0000 7f8597ba7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8590112d90 con 0x7f8590102af0 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45888/0 (socket says 172.21.15.37:45888) 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 -- 172.21.15.37:0/2064478199 learned_addr learned my addr 172.21.15.37:0/2064478199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8595943700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590102af0 0x7f85901990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8595142700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85901034b0 0x7f85901995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 -- 172.21.15.37:0/2064478199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590102af0 msgr2=0x7f85901990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590102af0 0x7f85901990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 -- 172.21.15.37:0/2064478199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85901034b0 msgr2=0x7f85901995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85901034b0 0x7f85901995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 -- 172.21.15.37:0/2064478199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859019e0b0 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8596144700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f85800047b0 tx=0x7f8580005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580026070 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8580004c90 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8580003cd0 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859019e340 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.019+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a5c20 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.020+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590069730 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.022+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8580029070 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.022+0000 7f858effd700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 0x7f85780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.022+0000 7f8595943700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 0x7f85780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.022+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f85800d1070 con 0x7f8590108a80 2024-04-25T21:18:27.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.023+0000 7f8595943700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 0x7f85780a6270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8588009550 tx=0x7f85880091b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:27.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.027+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f858005dff0 con 0x7f8590108a80 2024-04-25T21:18:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.286+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859002df80 con 0x7f8590108a80 2024-04-25T21:18:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.287+0000 7f858effd700 1 -- 172.21.15.37:0/2064478199 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f8580013070 con 0x7f8590108a80 2024-04-25T21:18:27.287 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:18:27.288 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 msgr2=0x7f85780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 0x7f85780a6270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8588009550 tx=0x7f85880091b0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 msgr2=0x7f859019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f85800047b0 tx=0x7f8580005e80 comp rx=0 tx=0).stop 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 shutdown_connections 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f85780a3db0 0x7f85780a6270 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8590108a80 0x7f859019d970 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8590102af0 0x7f85901990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 --2- 172.21.15.37:0/2064478199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85901034b0 0x7f85901995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 >> 172.21.15.37:0/2064478199 conn(0x7f8590079b20 msgr2=0x7f8590101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 shutdown_connections 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:27.289+0000 7f8597ba7700 1 -- 172.21.15.37:0/2064478199 wait complete. 2024-04-25T21:18:27.291 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:18:27.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:27 smithi037 ceph-mon[29195]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:27.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:27 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2064478199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:27.897 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 84.3 --await-for 42.0' 2024-04-25T21:18:27.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 84.3 --await-for 42.0 2024-04-25T21:18:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:27 smithi177 ceph-mon[33043]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:27 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2064478199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:28.162 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:18:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:27 smithi098 ceph-mon[32569]: pgmap v310: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:27 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2064478199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:29.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3396153236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0108a30 msgr2=0x7fa8c0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3396153236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0108a30 0x7fa8c0108e10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0099c0 tx=0x7fa8bc009cd0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.899 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:29 smithi037 ceph-mon[29195]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3396153236 shutdown_connections 2024-04-25T21:18:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3396153236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c01034b0 0x7fa8c010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3396153236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0108a30 0x7fa8c0108e10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3396153236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c0102af0 0x7fa8c0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3396153236 >> 172.21.15.37:0/3396153236 conn(0x7fa8c0079b10 msgr2=0x7fa8c0079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3396153236 shutdown_connections 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3396153236 wait complete. 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.898+0000 7fa8c73b7700 1 Processor -- start 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 -- start start 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c01034b0 0x7fa8c0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c0108a30 0x7fa8c019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0112c00 con 0x7fa8c0102af0 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0112a80 con 0x7fa8c01034b0 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c73b7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c0112d80 con 0x7fa8c0108a30 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54924/0 (socket says 172.21.15.37:54924) 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 -- 172.21.15.37:0/3816947936 learned_addr learned my addr 172.21.15.37:0/3816947936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c4952700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c01034b0 0x7fa8c0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5954700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c0108a30 0x7fa8c019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 -- 172.21.15.37:0/3816947936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c0108a30 msgr2=0x7fa8c019d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c0108a30 0x7fa8c019d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 -- 172.21.15.37:0/3816947936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c01034b0 msgr2=0x7fa8c0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c01034b0 0x7fa8c0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.899+0000 7fa8c5153700 1 -- 172.21.15.37:0/3816947936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c019def0 con 0x7fa8c0102af0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0098e0 tx=0x7fa8bc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8b67fc700 1 -- 172.21.15.37:0/3816947936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc027070 con 0x7fa8c0102af0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c019e180 con 0x7fa8c0102af0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8b67fc700 1 -- 172.21.15.37:0/3816947936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc003f20 con 0x7fa8c0102af0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8b67fc700 1 -- 172.21.15.37:0/3816947936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc022d70 con 0x7fa8c0102af0 2024-04-25T21:18:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.900+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c01a5a60 con 0x7fa8c0102af0 2024-04-25T21:18:29.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.902+0000 7fa8b67fc700 1 -- 172.21.15.37:0/3816947936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8bc02a070 con 0x7fa8c0102af0 2024-04-25T21:18:29.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.903+0000 7fa8b67fc700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 0x7fa8ac0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.903+0000 7fa8b67fc700 1 -- 172.21.15.37:0/3816947936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fa8bc0d15f0 con 0x7fa8c0102af0 2024-04-25T21:18:29.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.903+0000 7fa8c4952700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 0x7fa8ac0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.903+0000 7fa8c4952700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 0x7fa8ac0a6310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8c007a0a0 tx=0x7fa8b000a380 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 Processor -- start 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 -- start start 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd18d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd1b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560adbc60ec0 con 0x560adbd0da80 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560adbc07af0 con 0x560adbd0e3a0 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560adbc5ce80 con 0x560adbd19270 2024-04-25T21:18:29.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54940/0 (socket says 172.21.15.37:54940) 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.904+0000 7fa8c5153700 1 -- 172.21.15.37:0/3463287533 learned_addr learned my addr 172.21.15.37:0/3463287533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5954700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd1b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c4952700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd18d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 -- 172.21.15.37:0/3463287533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 msgr2=0x560adbd1b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd1b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 -- 172.21.15.37:0/3463287533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 msgr2=0x560adbd18d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd18d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 -- 172.21.15.37:0/3463287533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adbbaea60 con 0x560adbd0da80 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c4952700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd18d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8c5153700 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc099a80 tx=0x7fa8bc0228a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b043288153ebca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8ab7fe700 1 -- 172.21.15.37:0/3463287533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc09d620 con 0x560adbd0da80 2024-04-25T21:18:29.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8ab7fe700 1 -- 172.21.15.37:0/3463287533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc0a19d0 con 0x560adbd0da80 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.905+0000 7fa8ab7fe700 1 -- 172.21.15.37:0/3463287533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0a5d70 con 0x560adbd0da80 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/3463287533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 msgr2=0x560adbd0de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc099a80 tx=0x7fa8bc0228a0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/3463287533 shutdown_connections 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd1b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd0de60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/3463287533 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd18d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/3463287533 >> 172.21.15.37:0/3463287533 conn(0x560adbc3b820 msgr2=0x560adbd09f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/3463287533 shutdown_connections 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/3463287533 wait complete. 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 -- start start 2024-04-25T21:18:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.906+0000 7fa8d1c40b80 1 Processor -- start 2024-04-25T21:18:29.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd15db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560adbc60ec0 con 0x560adbd0da80 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560adbbffd60 con 0x560adbd0e3a0 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560adbd1bba0 con 0x560adbd19270 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8d1c40b80 10 client.0 _ll_register_callbacks cb 0x560adbbf9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54944/0 (socket says 172.21.15.37:54944) 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 -- 172.21.15.37:0/1964621620 learned_addr learned my addr 172.21.15.37:0/1964621620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5954700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c4952700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd15db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 -- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 msgr2=0x560adbd162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 -- 172.21.15.37:0/1964621620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 msgr2=0x560adbd15db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd15db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 -- 172.21.15.37:0/1964621620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adbc18400 con 0x560adbd0da80 2024-04-25T21:18:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0880b0 tx=0x7fa8bc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf3b6f76b154105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.907+0000 7fa8a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0bb060 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fa8bc09b920 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8bc0a19d0 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8bc0a6200 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 10 client.16539 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560adbd262e0 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc0a19d0 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8bc0bc260 con 0x560adbd0da80 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 10 client.16539 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 10 client.16539 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 20 client.16539 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd287b0 0x560adbd2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 4 client.16539 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.908+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560adbc6e6b0 con 0x560adbd287b0 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.909+0000 7fa8c4952700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd287b0 0x560adbd2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.909+0000 7fa8c4952700 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd287b0 0x560adbd2aba0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba101756849ace8c server_cookie=d4d7d10076591675 in_seq=0 out_seq=0 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.909+0000 7fa8a8ff9700 10 client.16539 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/1964621620 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560adbc6e6b0 con 0x560adbd287b0 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8a8ff9700 10 client.16539 handle_command_reply: tid=0 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 client.16539 shutdown 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 msgr2=0x560adbd18ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0880b0 tx=0x7fa8bc009710 comp rx=0 tx=0).stop 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 shutdown_connections 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd287b0 0x560adbd2aba0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd18ab0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1964621620 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd15db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 >> 172.21.15.37:0/1964621620 conn(0x560adbc3b820 msgr2=0x560adbd09b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8a8ff9700 0 client.16539 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 shutdown_connections 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1964621620 wait complete. 2024-04-25T21:18:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.910+0000 7fa8d1c40b80 20 client.16539 trim_cache size 0 max 16384 2024-04-25T21:18:29.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 Processor -- start 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 -- start start 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd30740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd13030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01ad000 con 0x560adbd0da80 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01aeb40 con 0x560adbd0e3a0 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01ad6d0 con 0x560adbd19270 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54946/0 (socket says 172.21.15.37:54946) 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 -- 172.21.15.37:0/1798191795 learned_addr learned my addr 172.21.15.37:0/1798191795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:29.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5954700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd13030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c4952700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd30740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 -- 172.21.15.37:0/1798191795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 msgr2=0x560adbd13030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd13030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 -- 172.21.15.37:0/1798191795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 msgr2=0x560adbd30740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd30740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5153700 1 -- 172.21.15.37:0/1798191795 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560adbd18e10 con 0x560adbd0da80 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c4952700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd30740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.930+0000 7fa8c5954700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd13030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8c5153700 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc099a80 tx=0x7fa8bc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8ca55f574c496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8aaffd700 1 -- 172.21.15.37:0/1798191795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0bb070 con 0x560adbd0da80 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8aaffd700 1 -- 172.21.15.37:0/1798191795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc0a19d0 con 0x560adbd0da80 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8aaffd700 1 -- 172.21.15.37:0/1798191795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc005140 con 0x560adbd0da80 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1798191795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 msgr2=0x560adbd30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc099a80 tx=0x7fa8bc009310 comp rx=0 tx=0).stop 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1798191795 shutdown_connections 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd19270 0x560adbd13030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd0da80 0x560adbd30200 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/1798191795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0e3a0 0x560adbd30740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1798191795 >> 172.21.15.37:0/1798191795 conn(0x560adbc3b820 msgr2=0x560adbd17e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1798191795 shutdown_connections 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.931+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/1798191795 wait complete. 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 -- start start 2024-04-25T21:18:29.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 Processor -- start 2024-04-25T21:18:29.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0da80 0x560adbd20a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd0e3a0 0x560adbd20f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01ae470 con 0x560adbd19270 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01adda0 con 0x560adbd0da80 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c01aafc0 con 0x560adbd0e3a0 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8d1c40b80 10 client.0 _ll_register_callbacks cb 0x560adbbf9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8c5954700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8c5954700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54952/0 (socket says 172.21.15.37:54952) 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.932+0000 7fa8c5954700 1 -- 172.21.15.37:0/2305080858 learned_addr learned my addr 172.21.15.37:0/2305080858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c4952700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd0e3a0 0x560adbd20f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5153700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0da80 0x560adbd20a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 -- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd0e3a0 msgr2=0x560adbd20f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd0e3a0 0x560adbd20f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 -- 172.21.15.37:0/2305080858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0da80 msgr2=0x560adbd20a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:29.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0da80 0x560adbd20a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 -- 172.21.15.37:0/2305080858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560adbd1f610 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8c5954700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8017fc0 tx=0x7fa8b8015410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725b69bed58b0bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8017320 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fa8b8004bb0 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8bc0a19d0 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8b8004db0 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8d1c40b80 10 client.16545 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.933+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560adbd8f550 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc0a19d0 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8b801e030 con 0x560adbd19270 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 10 client.16545 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:29.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 10 client.16545 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 20 client.16545 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd8fd80 0x560adbd921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 4 client.16545 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.3, "await_for": 42.0}] 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.3, "await_for": 42.0}) -- 0x560adbc68750 con 0x560adbd8fd80 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8c5153700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd8fd80 0x560adbd921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.934+0000 7fa8c5153700 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd8fd80 0x560adbd921a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcab6c1b01e283a5 server_cookie=c721a86048e7287f in_seq=0 out_seq=0 2024-04-25T21:18:29.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.935+0000 7fa8a8ff9700 10 client.16545 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8a8ff9700 1 -- 172.21.15.37:0/2305080858 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560adbc68750 con 0x560adbd8fd80 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8a8ff9700 10 client.16545 handle_command_reply: tid=0 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 client.16545 shutdown 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 msgr2=0x560adbd1ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8017fc0 tx=0x7fa8b8015410 comp rx=0 tx=0).stop 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 shutdown_connections 2024-04-25T21:18:29.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560adbd8fd80 0x560adbd921a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560adbd19270 0x560adbd1ef40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560adbd0da80 0x560adbd20a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8a8ff9700 0 client.16545 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 --2- 172.21.15.37:0/2305080858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560adbd0e3a0 0x560adbd20f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 >> 172.21.15.37:0/2305080858 conn(0x560adbc3b820 msgr2=0x560adbd175e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 shutdown_connections 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 1 -- 172.21.15.37:0/2305080858 wait complete. 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.950+0000 7fa8d1c40b80 20 client.16545 trim_cache size 0 max 16384 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:18:29.951 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 26, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "308523c7": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 26, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.3, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:18:29.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:18:29.953 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 msgr2=0x7fa8ac0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 0x7fa8ac0a6310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa8c007a0a0 tx=0x7fa8b000a380 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 msgr2=0x7fa8c0198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0098e0 tx=0x7fa8bc0057c0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 shutdown_connections 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8ac0a3e50 0x7fa8ac0a6310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8c0108a30 0x7fa8c019d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c0102af0 0x7fa8c0198f30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 --2- 172.21.15.37:0/3816947936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8c01034b0 0x7fa8c0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 >> 172.21.15.37:0/3816947936 conn(0x7fa8c0079b10 msgr2=0x7fa8c0101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 shutdown_connections 2024-04-25T21:18:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:29.951+0000 7fa8c73b7700 1 -- 172.21.15.37:0/3816947936 wait complete. 2024-04-25T21:18:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:29 smithi177 ceph-mon[33043]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:30.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:29 smithi098 ceph-mon[32569]: pgmap v311: 97 pgs: 97 active+clean; 5.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:18:30.752 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '308523c7', quiesce took 6.0 seconds. Will release after: 24.3 2024-04-25T21:18:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:31 smithi177 ceph-mon[33043]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:18:32.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:31 smithi098 ceph-mon[32569]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:18:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:31 smithi037 ceph-mon[29195]: pgmap v312: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-25T21:18:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:33 smithi177 ceph-mon[33043]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:18:34.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:33 smithi098 ceph-mon[32569]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:18:34.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:33 smithi037 ceph-mon[29195]: pgmap v313: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:18:35.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 -- 172.21.15.37:0/4199030423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 msgr2=0x7f46ac10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 --2- 172.21.15.37:0/4199030423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac10dd40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009ca0 tx=0x7f46a402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 -- 172.21.15.37:0/4199030423 shutdown_connections 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 --2- 172.21.15.37:0/4199030423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac10dd40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 --2- 172.21.15.37:0/4199030423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac101aa0 0x7f46ac105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 --2- 172.21.15.37:0/4199030423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac1010f0 0x7f46ac1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 -- 172.21.15.37:0/4199030423 >> 172.21.15.37:0/4199030423 conn(0x7f46ac077fd0 msgr2=0x7f46ac0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 -- 172.21.15.37:0/4199030423 shutdown_connections 2024-04-25T21:18:35.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.504+0000 7f46b10a7640 1 -- 172.21.15.37:0/4199030423 wait complete. 2024-04-25T21:18:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.505+0000 7f46b10a7640 1 Processor -- start 2024-04-25T21:18:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.505+0000 7f46b10a7640 1 -- start start 2024-04-25T21:18:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac1010f0 0x7f46ac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac19cf80 0x7f46ac19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1110e0 con 0x7f46ac106170 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac110f60 con 0x7f46ac19cf80 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46b10a7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac111260 con 0x7f46ac1010f0 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54974/0 (socket says 172.21.15.37:54974) 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 -- 172.21.15.37:0/301277791 learned_addr learned my addr 172.21.15.37:0/301277791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46ab59e640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac19cf80 0x7f46ac19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aad9d640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac1010f0 0x7f46ac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 -- 172.21.15.37:0/301277791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac1010f0 msgr2=0x7f46ac1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac1010f0 0x7f46ac1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 -- 172.21.15.37:0/301277791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac19cf80 msgr2=0x7f46ac19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac19cf80 0x7f46ac19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 -- 172.21.15.37:0/301277791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac19da70 con 0x7f46ac106170 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.506+0000 7f46aa59c640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46a000e9f0 tx=0x7f46a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.507+0000 7f4693fff640 1 -- 172.21.15.37:0/301277791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a000ccc0 con 0x7f46ac106170 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.507+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac19dd60 con 0x7f46ac106170 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.507+0000 7f4693fff640 1 -- 172.21.15.37:0/301277791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a00054d0 con 0x7f46ac106170 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.507+0000 7f4693fff640 1 -- 172.21.15.37:0/301277791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0010430 con 0x7f46ac106170 2024-04-25T21:18:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.507+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac1a5500 con 0x7f46ac106170 2024-04-25T21:18:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.509+0000 7f4693fff640 1 -- 172.21.15.37:0/301277791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f46a00046c0 con 0x7f46ac106170 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.509+0000 7f46b3dc2740 1 Processor -- start 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 -- start start 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460520 0x557f5a460900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a472000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f5a3c5970 con 0x557f5a460e40 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f5a3624a0 con 0x557f5a46fc10 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557f5a3c1930 con 0x557f5a460520 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f4693fff640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 0x7f46940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54976/0 (socket says 172.21.15.37:54976) 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 -- 172.21.15.37:0/504315986 learned_addr learned my addr 172.21.15.37:0/504315986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aa59c640 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a472000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46ab59e640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 0x7f46940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46ab59e640 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460520 0x557f5a460900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f4693fff640 1 -- 172.21.15.37:0/301277791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f46a00d0b30 con 0x7f46ac106170 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 -- 172.21.15.37:0/504315986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460520 msgr2=0x557f5a460900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460520 0x557f5a460900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 -- 172.21.15.37:0/504315986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 msgr2=0x557f5a472000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a472000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.510+0000 7f46aad9d640 1 -- 172.21.15.37:0/504315986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5a0d8210 con 0x557f5a460e40 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46aad9d640 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f469c009cf0 tx=0x7f469c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e9797576a59f4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46ab59e640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 0x7f46940a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46a402f500 tx=0x7f46a402f310 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46917fa640 1 -- 172.21.15.37:0/504315986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004030 con 0x557f5a460e40 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46917fa640 1 -- 172.21.15.37:0/504315986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0041d0 con 0x557f5a460e40 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46917fa640 1 -- 172.21.15.37:0/504315986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c019710 con 0x557f5a460e40 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46b3dc2740 1 -- 172.21.15.37:0/504315986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 msgr2=0x557f5a46f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.511+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f469c009cf0 tx=0x7f469c011810 comp rx=0 tx=0).stop 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- 172.21.15.37:0/504315986 shutdown_connections 2024-04-25T21:18:35.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a472000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46f6d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/504315986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460520 0x557f5a460900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- 172.21.15.37:0/504315986 >> 172.21.15.37:0/504315986 conn(0x557f5a45cb30 msgr2=0x557f5a45cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- 172.21.15.37:0/504315986 shutdown_connections 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- 172.21.15.37:0/504315986 wait complete. 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- start start 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 Processor -- start 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a46f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 0x557f5a46cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557f5a472840 con 0x557f5a460e40 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x557f5a3600b0 con 0x557f5a460520 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x557f5a4729c0 con 0x557f5a46fc10 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46b3dc2740 10 client.0 _ll_register_callbacks cb 0x557f5a34dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46aa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46aa59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54982/0 (socket says 172.21.15.37:54982) 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.512+0000 7f46aa59c640 1 -- 172.21.15.37:0/1823447854 learned_addr learned my addr 172.21.15.37:0/1823447854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46ab59e640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 0x557f5a46cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a46f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 -- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 msgr2=0x557f5a46cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 0x557f5a46cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 -- 172.21.15.37:0/1823447854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 msgr2=0x557f5a46f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a46f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 -- 172.21.15.37:0/1823447854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5a1a4620 con 0x557f5a460e40 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46ab59e640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 0x557f5a46cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a46f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46a009ca60 tx=0x7f46a006e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b109cbdf328aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f468eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:35.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a009cad0 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f46a0099140 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469c0041d0 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f46a00a0e80 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46b3dc2740 10 client.16557 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f5a47e790 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.513+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0041d0 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f46a006e110 con 0x557f5a460e40 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 10 client.16557 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 20 client.16557 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a47fc00 0x557f5a481ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 4 client.16557 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f5a3d3390 con 0x557f5a47fc00 2024-04-25T21:18:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a47fc00 0x557f5a481ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a47fc00 0x557f5a481ff0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce0e097abf07c47 server_cookie=3f5596ed9afa2f10 in_seq=0 out_seq=0 2024-04-25T21:18:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.514+0000 7f468eff5640 10 client.16557 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f468eff5640 1 -- 172.21.15.37:0/1823447854 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f5a3d3390 con 0x557f5a47fc00 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f468eff5640 10 client.16557 handle_command_reply: tid=0 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 client.16557 shutdown 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 msgr2=0x557f5a46c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46a009ca60 tx=0x7f46a006e510 comp rx=0 tx=0).stop 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 shutdown_connections 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a47fc00 0x557f5a481ff0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a46fc10 0x557f5a46cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460e40 0x557f5a46c7f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f468eff5640 0 client.16557 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1823447854 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a46f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 >> 172.21.15.37:0/1823447854 conn(0x557f5a45cb30 msgr2=0x557f5a4629a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.515+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 shutdown_connections 2024-04-25T21:18:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.516+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1823447854 wait complete. 2024-04-25T21:18:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.516+0000 7f46b3dc2740 20 client.16557 trim_cache size 0 max 16384 2024-04-25T21:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 Processor -- start 2024-04-25T21:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 -- start start 2024-04-25T21:18:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a47b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a467d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1ab900 con 0x557f5a46fc10 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1ad440 con 0x557f5a460520 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1abfd0 con 0x557f5a460e40 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54998/0 (socket says 172.21.15.37:54998) 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 -- 172.21.15.37:0/2014929475 learned_addr learned my addr 172.21.15.37:0/2014929475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46aad9d640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a47b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46aa59c640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a467d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 -- 172.21.15.37:0/2014929475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 msgr2=0x557f5a467d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a467d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 -- 172.21.15.37:0/2014929475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 msgr2=0x557f5a47b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a47b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 -- 172.21.15.37:0/2014929475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5a45c960 con 0x557f5a46fc10 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46aa59c640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a467d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.531+0000 7f46ab59e640 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46a4035000 tx=0x7f46a403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af792a03b9a4a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f4690ff9640 1 -- 172.21.15.37:0/2014929475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4031450 con 0x557f5a46fc10 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f4690ff9640 1 -- 172.21.15.37:0/2014929475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a4003970 con 0x557f5a46fc10 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f4690ff9640 1 -- 172.21.15.37:0/2014929475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a40045c0 con 0x557f5a46fc10 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 -- 172.21.15.37:0/2014929475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 msgr2=0x557f5a46f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46a4035000 tx=0x7f46a403f040 comp rx=0 tx=0).stop 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 -- 172.21.15.37:0/2014929475 shutdown_connections 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a46fc10 0x557f5a46f030 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a467d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/2014929475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a460520 0x557f5a47b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 -- 172.21.15.37:0/2014929475 >> 172.21.15.37:0/2014929475 conn(0x557f5a45cb30 msgr2=0x557f5a467670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 -- 172.21.15.37:0/2014929475 shutdown_connections 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.532+0000 7f46b3dc2740 1 -- 172.21.15.37:0/2014929475 wait complete. 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 -- start start 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 Processor -- start 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a469980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a469ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1acd70 con 0x557f5a460520 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1ac6a0 con 0x557f5a46fc10 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1a9900 con 0x557f5a460e40 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46b3dc2740 10 client.0 _ll_register_callbacks cb 0x7f46a009c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55006/0 (socket says 172.21.15.37:55006) 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 -- 172.21.15.37:0/1913038954 learned_addr learned my addr 172.21.15.37:0/1913038954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46ab59e640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a469ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a469980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 -- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 msgr2=0x557f5a469980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a469980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 -- 172.21.15.37:0/1913038954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 msgr2=0x557f5a469ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a469ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 -- 172.21.15.37:0/1913038954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5a4e5120 con 0x557f5a460520 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a469980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46ab59e640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a469ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f46aad9d640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f469c009cf0 tx=0x7f469c013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ff52bf7928b0b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.533+0000 7f468eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c030070 con 0x557f5a460520 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f469c005040 con 0x557f5a460520 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a4003970 con 0x557f5a460520 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f469c03c050 con 0x557f5a460520 2024-04-25T21:18:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 10 client.16563 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f5a4e6310 con 0x557f5a460520 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a4003970 con 0x557f5a460520 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f469c02c730 con 0x557f5a460520 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 10 client.16563 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 20 client.16563 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a4e6930 0x557f5a4e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 4 client.16563 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f5a46d0c0 con 0x557f5a4e6930 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.534+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a4e6930 0x557f5a4e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.535+0000 7f46aa59c640 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a4e6930 0x557f5a4e8d50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0bd8b7a1559a3e server_cookie=e5a997a7aba44aa1 in_seq=0 out_seq=0 2024-04-25T21:18:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.535+0000 7f468eff5640 10 client.16563 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f468eff5640 1 -- 172.21.15.37:0/1913038954 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f5a46d0c0 con 0x557f5a4e6930 2024-04-25T21:18:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f468eff5640 10 client.16563 handle_command_reply: tid=0 2024-04-25T21:18:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 client.16563 shutdown 2024-04-25T21:18:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 msgr2=0x557f5a469440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f469c009cf0 tx=0x7f469c013f40 comp rx=0 tx=0).stop 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 shutdown_connections 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x557f5a4e6930 0x557f5a4e8d50 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x557f5a46fc10 0x557f5a469ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557f5a460e40 0x557f5a469980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f468eff5640 0 client.16563 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 --2- 172.21.15.37:0/1913038954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557f5a460520 0x557f5a469440 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 >> 172.21.15.37:0/1913038954 conn(0x557f5a45cb30 msgr2=0x557f5a45b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 shutdown_connections 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 1 -- 172.21.15.37:0/1913038954 wait complete. 2024-04-25T21:18:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.550+0000 7f46b3dc2740 20 client.16563 trim_cache size 0 max 16384 2024-04-25T21:18:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:18:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:18:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:18:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:18:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 msgr2=0x7f46940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 0x7f46940a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46a402f500 tx=0x7f46a402f310 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 msgr2=0x7f46ac198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46a000e9f0 tx=0x7f46a000eeb0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 shutdown_connections 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46ac19cf80 0x7f46ac19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46ac106170 0x7f46ac198c00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f46940a3dc0 0x7f46940a6280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 --2- 172.21.15.37:0/301277791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46ac1010f0 0x7f46ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 >> 172.21.15.37:0/301277791 conn(0x7f46ac077fd0 msgr2=0x7f46ac0698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 shutdown_connections 2024-04-25T21:18:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:35.551+0000 7f46b10a7640 1 -- 172.21.15.37:0/301277791 wait complete. 2024-04-25T21:18:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:35 smithi177 ceph-mon[33043]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-25T21:18:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:35 smithi098 ceph-mon[32569]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-25T21:18:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:35 smithi037 ceph-mon[29195]: pgmap v314: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-25T21:18:38.037 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:37 smithi037 ceph-mon[29195]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:37 smithi177 ceph-mon[33043]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:38.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:37 smithi098 ceph-mon[32569]: pgmap v315: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:39 smithi177 ceph-mon[33043]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:39 smithi098 ceph-mon[32569]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:39 smithi037 ceph-mon[29195]: pgmap v316: 97 pgs: 97 active+clean; 5.4 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T21:18:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:41 smithi177 ceph-mon[33043]: pgmap v317: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-04-25T21:18:42.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:41 smithi098 ceph-mon[32569]: pgmap v317: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-04-25T21:18:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:41 smithi037 ceph-mon[29195]: pgmap v317: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 19 op/s 2024-04-25T21:18:43.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:42 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:18:43.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:42 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:18:43.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:42 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: pgmap v318: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:43 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: pgmap v318: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T21:18:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:43.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:43 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: pgmap v318: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:43 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: pgmap v319: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:18:45.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: pgmap v319: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:18:45.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: pgmap v319: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-25T21:18:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:18:45.986 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:18:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:18:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:18:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:18:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:18:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:18:46.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:46 smithi177 ceph-mon[33043]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:18:46.901 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:46 smithi098 ceph-mon[32569]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:18:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi177 to 176.1M 2024-04-25T21:18:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi177 to 184727961: error parsing value: Value '184727961' is below minimum 939524096 2024-04-25T21:18:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi098 to 176.2M 2024-04-25T21:18:46.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi098 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:18:46.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Adjusting osd_memory_target on smithi037 to 177.2M 2024-04-25T21:18:46.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:46 smithi037 ceph-mon[29195]: Unable to set osd_memory_target on smithi037 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-25T21:18:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:47 smithi177 ceph-mon[33043]: pgmap v320: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:47.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:47 smithi098 ceph-mon[32569]: pgmap v320: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:47 smithi037 ceph-mon[29195]: pgmap v320: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- 172.21.15.37:0/3622993957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 msgr2=0x7f6c4c1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 --2- 172.21.15.37:0/3622993957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c1014c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c4802a410 comp rx=0 tx=0).stop 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- 172.21.15.37:0/3622993957 shutdown_connections 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 --2- 172.21.15.37:0/3622993957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 0x7f6c4c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 --2- 172.21.15.37:0/3622993957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c101a90 0x7f6c4c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 --2- 172.21.15.37:0/3622993957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c1014c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- 172.21.15.37:0/3622993957 >> 172.21.15.37:0/3622993957 conn(0x7f6c4c0771d0 msgr2=0x7f6c4c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- 172.21.15.37:0/3622993957 shutdown_connections 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- 172.21.15.37:0/3622993957 wait complete. 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 Processor -- start 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.651+0000 7f6c53f07640 1 -- start start 2024-04-25T21:18:48.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 0x7f6c4c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c197d80 0x7f6c4c1a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c111110 con 0x7f6c4c1010e0 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c110f90 con 0x7f6c4c1061b0 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c53f07640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c111290 con 0x7f6c4c197d80 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54594/0 (socket says 172.21.15.37:54594) 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 -- 172.21.15.37:0/214658614 learned_addr learned my addr 172.21.15.37:0/214658614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c514a2640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 0x7f6c4c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c524a4640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c197d80 0x7f6c4c1a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 -- 172.21.15.37:0/214658614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c197d80 msgr2=0x7f6c4c1a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c197d80 0x7f6c4c1a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 -- 172.21.15.37:0/214658614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 msgr2=0x7f6c4c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 0x7f6c4c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 -- 172.21.15.37:0/214658614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c1a6f60 con 0x7f6c4c1010e0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.652+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000c00 tx=0x7f6c48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.653+0000 7f6c42ffd640 1 -- 172.21.15.37:0/214658614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4803f030 con 0x7f6c4c1010e0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.653+0000 7f6c42ffd640 1 -- 172.21.15.37:0/214658614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c48003aa0 con 0x7f6c4c1010e0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.653+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a71f0 con 0x7f6c4c1010e0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.653+0000 7f6c42ffd640 1 -- 172.21.15.37:0/214658614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48035070 con 0x7f6c4c1010e0 2024-04-25T21:18:48.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.653+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a7730 con 0x7f6c4c1010e0 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.655+0000 7f6c42ffd640 1 -- 172.21.15.37:0/214658614 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c48042030 con 0x7f6c4c1010e0 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.655+0000 7f6c56c16740 1 Processor -- start 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 -- start start 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c42ffd640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 0x7f6c380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f0822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f082800 0x55fe4f084bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fe4efdc970 con 0x55fe4f077620 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fe4ef794a0 con 0x55fe4f082800 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c56c16740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fe4efd8930 con 0x55fe4f077f40 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c524a4640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 0x7f6c380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54600/0 (socket says 172.21.15.37:54600) 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 -- 172.21.15.37:0/49478378 learned_addr learned my addr 172.21.15.37:0/49478378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c42ffd640 1 -- 172.21.15.37:0/214658614 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6c480f0e60 con 0x7f6c4c1010e0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c514a2640 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f0822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c524a4640 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f082800 0x55fe4f084bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 -- 172.21.15.37:0/49478378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 msgr2=0x55fe4f0822c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f0822c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 -- 172.21.15.37:0/49478378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f082800 msgr2=0x55fe4f084bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f082800 0x55fe4f084bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.656+0000 7f6c51ca3640 1 -- 172.21.15.37:0/49478378 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4ecef210 con 0x55fe4f077620 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.657+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ee5b0 tx=0x7f6c48036b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bf51b48620b873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.657+0000 7f6c524a4640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 0x7f6c380a6300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009370 tx=0x7f6c44013ae0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.657+0000 7f6c37fff640 1 -- 172.21.15.37:0/49478378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48049070 con 0x55fe4f077620 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.657+0000 7f6c37fff640 1 -- 172.21.15.37:0/49478378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c480bd4a0 con 0x55fe4f077620 2024-04-25T21:18:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.657+0000 7f6c37fff640 1 -- 172.21.15.37:0/49478378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480c5a10 con 0x55fe4f077620 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- 172.21.15.37:0/49478378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 msgr2=0x55fe4f077a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ee5b0 tx=0x7f6c48036b60 comp rx=0 tx=0).stop 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- 172.21.15.37:0/49478378 shutdown_connections 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f082800 0x55fe4f084bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f0822c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 --2- 172.21.15.37:0/49478378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077620 0x55fe4f077a00 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- 172.21.15.37:0/49478378 >> 172.21.15.37:0/49478378 conn(0x55fe4f073b30 msgr2=0x55fe4f073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- 172.21.15.37:0/49478378 shutdown_connections 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- 172.21.15.37:0/49478378 wait complete. 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 -- start start 2024-04-25T21:18:48.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.658+0000 7f6c56c16740 1 Processor -- start 2024-04-25T21:18:48.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f07f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fe4efdc970 con 0x55fe4f082800 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55fe4ef770b0 con 0x55fe4f077620 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55fe4f0851f0 con 0x55fe4f077f40 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c56c16740 10 client.0 _ll_register_callbacks cb 0x55fe4ef64d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c524a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c524a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54608/0 (socket says 172.21.15.37:54608) 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:53226/0 (socket says 172.21.15.37:53226) 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c524a4640 1 -- 172.21.15.37:0/1991809246 learned_addr learned my addr 172.21.15.37:0/1991809246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f07f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 -- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 msgr2=0x55fe4f07f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f07f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 -- 172.21.15.37:0/1991809246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 msgr2=0x55fe4f07fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 -- 172.21.15.37:0/1991809246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4edbb620 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c524a4640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ee5b0 tx=0x7f6c480bcda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d582571d6a44b286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.659+0000 7f6c357fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480c0030 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c480bd4a0 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6c480b9410 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c48049070 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 10 client.25922 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe4f091000 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c480bd4a0 con 0x55fe4f077620 2024-04-25T21:18:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c48003c50 con 0x55fe4f077620 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 10 client.25922 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 10 client.25922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 20 client.25922 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0923a0 0x55fe4f094790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 4 client.25922 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.660+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe4efea390 con 0x55fe4f0923a0 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.661+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0923a0 0x55fe4f094790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.661+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0923a0 0x55fe4f094790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36c035a42f35ef5 server_cookie=98dd8dfb8fb5258b in_seq=0 out_seq=0 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.661+0000 7f6c357fa640 10 client.25922 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c357fa640 1 -- 172.21.15.37:0/1991809246 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe4efea390 con 0x55fe4f0923a0 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c357fa640 10 client.25922 handle_command_reply: tid=0 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 client.25922 shutdown 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 msgr2=0x55fe4f0821d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ee5b0 tx=0x7f6c480bcda0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 shutdown_connections 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0923a0 0x55fe4f094790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f07f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1991809246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f0821d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c357fa640 0 client.25922 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 >> 172.21.15.37:0/1991809246 conn(0x55fe4f073b30 msgr2=0x55fe4f0798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 shutdown_connections 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 1 -- 172.21.15.37:0/1991809246 wait complete. 2024-04-25T21:18:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.662+0000 7f6c56c16740 20 client.25922 trim_cache size 0 max 16384 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 Processor -- start 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 -- start start 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f073430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f08dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1afd30 con 0x55fe4f082800 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b1870 con 0x55fe4f077620 2024-04-25T21:18:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.677+0000 7f6c56c16740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b0400 con 0x55fe4f077f40 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54616/0 (socket says 172.21.15.37:54616) 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 -- 172.21.15.37:0/1162313209 learned_addr learned my addr 172.21.15.37:0/1162313209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 -- 172.21.15.37:0/1162313209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 msgr2=0x55fe4f08dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f08dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f08dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 -- 172.21.15.37:0/1162313209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 msgr2=0x55fe4f073430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f073430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 -- 172.21.15.37:0/1162313209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4f07eb50 con 0x55fe4f082800 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c524a4640 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c44013440 tx=0x7f6c4401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723f50db6caf6f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c377fe640 1 -- 172.21.15.37:0/1162313209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44020730 con 0x55fe4f082800 2024-04-25T21:18:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c377fe640 1 -- 172.21.15.37:0/1162313209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44020d80 con 0x55fe4f082800 2024-04-25T21:18:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.678+0000 7f6c377fe640 1 -- 172.21.15.37:0/1162313209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44012e10 con 0x55fe4f082800 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- 172.21.15.37:0/1162313209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 msgr2=0x55fe4f07dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6c44013440 tx=0x7f6c4401c000 comp rx=0 tx=0).stop 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- 172.21.15.37:0/1162313209 shutdown_connections 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f082800 0x55fe4f07dca0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f077f40 0x55fe4f08dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1162313209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- 172.21.15.37:0/1162313209 >> 172.21.15.37:0/1162313209 conn(0x55fe4f073b30 msgr2=0x55fe4f08d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- 172.21.15.37:0/1162313209 shutdown_connections 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- 172.21.15.37:0/1162313209 wait complete. 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 -- start start 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.679+0000 7f6c56c16740 1 Processor -- start 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f08a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f082800 0x55fe4f088200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b11a0 con 0x55fe4f077f40 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b0ad0 con 0x55fe4f077620 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1add30 con 0x55fe4f082800 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c56c16740 10 client.0 _ll_register_callbacks cb 0x7f6c48031950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54626/0 (socket says 172.21.15.37:54626) 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 -- 172.21.15.37:0/1097733579 learned_addr learned my addr 172.21.15.37:0/1097733579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f08a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c524a4640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f082800 0x55fe4f088200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 -- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f082800 msgr2=0x55fe4f088200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f082800 0x55fe4f088200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 -- 172.21.15.37:0/1097733579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 msgr2=0x55fe4f08a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f08a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 -- 172.21.15.37:0/1097733579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4f088a30 con 0x55fe4f077f40 2024-04-25T21:18:48.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c514a2640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d20 tx=0x7f6c3c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3eef00127e498a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c357fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.680+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c024020 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f6c3c020070 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c44020d80 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c02f050 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 10 client.16581 fetch_fsmap learned FSMap version 27 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fe4f0f8910 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44020d80 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6c3c0108f0 con 0x55fe4f077f40 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 10 client.16581 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 20 client.16581 populate_metadata read hostname 'smithi037' 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0f90c0 0x55fe4f0fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 4 client.16581 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe4efe4200 con 0x55fe4f0f90c0 2024-04-25T21:18:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.681+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0f90c0 0x55fe4f0fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.682+0000 7f6c51ca3640 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0f90c0 0x55fe4f0fb4e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b96660b4fb37ae7 server_cookie=43ff36d2c0d4dcb1 in_seq=0 out_seq=0 2024-04-25T21:18:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.682+0000 7f6c357fa640 10 client.16581 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.693+0000 7f6c357fa640 1 -- 172.21.15.37:0/1097733579 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe4efe4200 con 0x55fe4f0f90c0 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.693+0000 7f6c357fa640 10 client.16581 handle_command_reply: tid=0 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.693+0000 7f6c56c16740 1 client.16581 shutdown 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 msgr2=0x55fe4f087c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009d20 tx=0x7f6c3c009f90 comp rx=0 tx=0).stop 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 shutdown_connections 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55fe4f0f90c0 0x55fe4f0fb4e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fe4f082800 0x55fe4f088200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fe4f077f40 0x55fe4f087c80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 --2- 172.21.15.37:0/1097733579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55fe4f077620 0x55fe4f08a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c357fa640 0 client.16581 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 >> 172.21.15.37:0/1097733579 conn(0x55fe4f073b30 msgr2=0x55fe4f072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 shutdown_connections 2024-04-25T21:18:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 1 -- 172.21.15.37:0/1097733579 wait complete. 2024-04-25T21:18:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c56c16740 20 client.16581 trim_cache size 0 max 16384 2024-04-25T21:18:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:18:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:18:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:18:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 msgr2=0x7f6c380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 0x7f6c380a6300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009370 tx=0x7f6c44013ae0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 msgr2=0x7f6c4c197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6c48000c00 tx=0x7f6c48004270 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 shutdown_connections 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6c380a3e40 0x7f6c380a6300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c4c197d80 0x7f6c4c1a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c4c1061b0 0x7f6c4c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 --2- 172.21.15.37:0/214658614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c4c1010e0 0x7f6c4c197300 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.694+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 >> 172.21.15.37:0/214658614 conn(0x7f6c4c0771d0 msgr2=0x7f6c4c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.695+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 shutdown_connections 2024-04-25T21:18:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:18:48.695+0000 7f6c53f07640 1 -- 172.21.15.37:0/214658614 wait complete. 2024-04-25T21:18:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:49 smithi177 ceph-mon[33043]: pgmap v321: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:49 smithi098 ceph-mon[32569]: pgmap v321: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:49 smithi037 ceph-mon[29195]: pgmap v321: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:18:51.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:51 smithi177 ceph-mon[33043]: pgmap v322: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:18:51.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:51 smithi098 ceph-mon[32569]: pgmap v322: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:18:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:51 smithi037 ceph-mon[29195]: pgmap v322: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:18:53.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:53 smithi177 ceph-mon[33043]: pgmap v323: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T21:18:53.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:53 smithi098 ceph-mon[32569]: pgmap v323: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T21:18:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:53 smithi037 ceph-mon[29195]: pgmap v323: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-25T21:18:55.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:18:55.316 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:18:55.705 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:55 smithi037 ceph-mon[29195]: pgmap v324: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-25T21:18:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:55 smithi177 ceph-mon[33043]: pgmap v324: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-25T21:18:55.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:55 smithi098 ceph-mon[32569]: pgmap v324: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-25T21:18:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 -- 172.21.15.37:0/2727791809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d0108a60 msgr2=0x7f78d0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 --2- 172.21.15.37:0/2727791809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d0108a60 0x7f78d0108e40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f78cc009990 tx=0x7f78cc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 -- 172.21.15.37:0/2727791809 shutdown_connections 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 --2- 172.21.15.37:0/2727791809 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 --2- 172.21.15.37:0/2727791809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d0108a60 0x7f78d0108e40 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 --2- 172.21.15.37:0/2727791809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d006b1f0 0x7f78d006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 -- 172.21.15.37:0/2727791809 >> 172.21.15.37:0/2727791809 conn(0x7f78d00789e0 msgr2=0x7f78d0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 -- 172.21.15.37:0/2727791809 shutdown_connections 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.103+0000 7f78d78d9700 1 -- 172.21.15.37:0/2727791809 wait complete. 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 Processor -- start 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 -- start start 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d01a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 0x7f78d010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0112c60 con 0x7f78d006b1f0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0112ae0 con 0x7f78d006bbb0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.104+0000 7f78d78d9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0112de0 con 0x7f78d0108a60 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54656/0 (socket says 172.21.15.37:54656) 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 -- 172.21.15.37:0/1542262302 learned_addr learned my addr 172.21.15.37:0/1542262302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5e76700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 0x7f78d010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d4e74700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d01a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 -- 172.21.15.37:0/1542262302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 msgr2=0x7f78d010eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 0x7f78d010eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 -- 172.21.15.37:0/1542262302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 msgr2=0x7f78d01a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d01a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 -- 172.21.15.37:0/1542262302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d010f3a0 con 0x7f78d006b1f0 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5e76700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 0x7f78d010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d4e74700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d01a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:18:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d5675700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000c00 tx=0x7f78cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc027070 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d010f630 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78cc0041d0 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00f460 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.105+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01ac300 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.106+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d0069730 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.108+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78cc02a070 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.108+0000 7f78c67fc700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 0x7f78bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.108+0000 7f78d4e74700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 0x7f78bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.108+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f78cc0d1bf0 con 0x7f78d006b1f0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.109+0000 7f78d4e74700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 0x7f78bc0a6270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f78d010fe10 tx=0x7f78c0005f30 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:18:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.112+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78cc031070 con 0x7f78d006b1f0 2024-04-25T21:18:57.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.367+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d01a0b20 con 0x7f78d006b1f0 2024-04-25T21:18:57.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.368+0000 7f78c67fc700 1 -- 172.21.15.37:0/1542262302 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f78cc013070 con 0x7f78d006b1f0 2024-04-25T21:18:57.368 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:18:57.368 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:18:57.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 msgr2=0x7f78bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 0x7f78bc0a6270 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f78d010fe10 tx=0x7f78c0005f30 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 msgr2=0x7f78d01a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000c00 tx=0x7f78cc0047b0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 shutdown_connections 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f78bc0a3db0 0x7f78bc0a6270 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78d0108a60 0x7f78d010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78d006b1f0 0x7f78d01a54e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 --2- 172.21.15.37:0/1542262302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78d006bbb0 0x7f78d01a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 >> 172.21.15.37:0/1542262302 conn(0x7f78d00789e0 msgr2=0x7f78d0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 shutdown_connections 2024-04-25T21:18:57.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:18:57.370+0000 7f78d78d9700 1 -- 172.21.15.37:0/1542262302 wait complete. 2024-04-25T21:18:57.371 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:18:57.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:57 smithi037 ceph-mon[29195]: pgmap v325: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:18:57.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:57 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1542262302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:57 smithi177 ceph-mon[33043]: pgmap v325: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:18:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:57 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1542262302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:57.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:57 smithi098 ceph-mon[32569]: pgmap v325: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:18:57.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:57 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1542262302' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:18:58.079 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 308523c7 --release --await' 2024-04-25T21:18:58.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 308523c7 --release --await 2024-04-25T21:18:58.336 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:18:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:18:59 smithi177 ceph-mon[33043]: pgmap v326: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:18:59.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:18:59 smithi037 ceph-mon[29195]: pgmap v326: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:18:59.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:18:59 smithi098 ceph-mon[32569]: pgmap v326: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.122+0000 7fdccc45f700 1 -- 172.21.15.37:0/3914493432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4103450 msgr2=0x7fdcc410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.122+0000 7fdccc45f700 1 --2- 172.21.15.37:0/3914493432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4103450 0x7fdcc410f8b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b0a0 tx=0x7fdcbc00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 -- 172.21.15.37:0/3914493432 shutdown_connections 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- 172.21.15.37:0/3914493432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4103450 0x7fdcc410f8b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- 172.21.15.37:0/3914493432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4108a50 0x7fdcc4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- 172.21.15.37:0/3914493432 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 -- 172.21.15.37:0/3914493432 >> 172.21.15.37:0/3914493432 conn(0x7fdcc4079ba0 msgr2=0x7fdcc4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 -- 172.21.15.37:0/3914493432 shutdown_connections 2024-04-25T21:19:00.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 -- 172.21.15.37:0/3914493432 wait complete. 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 Processor -- start 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 -- start start 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4103450 0x7fdcc419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.123+0000 7fdccc45f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdccc45f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4112d30 con 0x7fdcc4108a50 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdccc45f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4112bb0 con 0x7fdcc4103450 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdccc45f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4112eb0 con 0x7fdcc4102a90 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39250/0 (socket says 172.21.15.37:39250) 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 -- 172.21.15.37:0/1920451909 learned_addr learned my addr 172.21.15.37:0/1920451909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4103450 0x7fdcc419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 -- 172.21.15.37:0/1920451909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 msgr2=0x7fdcc419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 -- 172.21.15.37:0/1920451909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4103450 msgr2=0x7fdcc419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4103450 0x7fdcc419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 -- 172.21.15.37:0/1920451909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc41a0530 con 0x7fdcc4108a50 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b070 tx=0x7fdcbc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcbb7fe700 1 -- 172.21.15.37:0/1920451909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc021070 con 0x7fdcc4108a50 2024-04-25T21:19:00.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcbb7fe700 1 -- 172.21.15.37:0/1920451909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcbc007cf0 con 0x7fdcc4108a50 2024-04-25T21:19:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc41a07c0 con 0x7fdcc4108a50 2024-04-25T21:19:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdcbb7fe700 1 -- 172.21.15.37:0/1920451909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc005440 con 0x7fdcc4108a50 2024-04-25T21:19:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.124+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a80a0 con 0x7fdcc4108a50 2024-04-25T21:19:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.127+0000 7fdcbb7fe700 1 -- 172.21.15.37:0/1920451909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdcbc024070 con 0x7fdcc4108a50 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.128+0000 7fdcbb7fe700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 0x7fdcb00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.128+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 0x7fdcb00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.128+0000 7fdcbb7fe700 1 -- 172.21.15.37:0/1920451909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fdcbc0d1ad0 con 0x7fdcc4108a50 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.128+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 0x7fdcb00a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00099d0 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.128+0000 7fdcd6ce8b80 1 Processor -- start 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 -- start start 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06382ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ad062cae30 con 0x55ad063779f0 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad06271a60 con 0x55ad063831e0 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ad062c6df0 con 0x55ad06378310 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39262/0 (socket says 172.21.15.37:39262) 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2532080460 learned_addr learned my addr 172.21.15.37:0/2532080460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06382ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2532080460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 msgr2=0x55ad06382ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06382ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2532080460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 msgr2=0x55ad063855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2532080460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad062189d0 con 0x55ad063779f0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.129+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.130+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06382ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.130+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d7c0 tx=0x7fdcc0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88e666797945119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.130+0000 7fdcb8d70700 1 -- 172.21.15.37:0/2532080460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0024070 con 0x55ad063779f0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.130+0000 7fdcb8d70700 1 -- 172.21.15.37:0/2532080460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc0014af0 con 0x55ad063779f0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.130+0000 7fdcb8d70700 1 -- 172.21.15.37:0/2532080460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc001fc30 con 0x55ad063779f0 2024-04-25T21:19:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2532080460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 msgr2=0x55ad06377dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000d7c0 tx=0x7fdcc0010000 comp rx=0 tx=0).stop 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2532080460 shutdown_connections 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06377dd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2532080460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06382ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2532080460 >> 172.21.15.37:0/2532080460 conn(0x55ad062a5790 msgr2=0x55ad06373ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2532080460 shutdown_connections 2024-04-25T21:19:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2532080460 wait complete. 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 -- start start 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.131+0000 7fdcd6ce8b80 1 Processor -- start 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad063779f0 0x55ad063829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ad06385bd0 con 0x55ad06378310 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ad06269cd0 con 0x55ad063831e0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ad06385d50 con 0x55ad063779f0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcd6ce8b80 10 client.0 _ll_register_callbacks cb 0x55ad06263a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39274/0 (socket says 172.21.15.37:39274) 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 -- 172.21.15.37:0/1004953871 learned_addr learned my addr 172.21.15.37:0/1004953871 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad063779f0 0x55ad063829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 -- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad063779f0 msgr2=0x55ad063829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad063779f0 0x55ad063829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 -- 172.21.15.37:0/1004953871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 msgr2=0x55ad063801e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063801e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 -- 172.21.15.37:0/1004953871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad06282370 con 0x55ad06378310 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400bf80 tx=0x7fdcb400f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fd31a0f9992266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.132+0000 7fdcadffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4018020 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fdcb4014460 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc0014af0 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcb4023050 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 10 client.16599 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad063904f0 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc0014af0 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcb4013730 con 0x55ad06378310 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 10 client.16599 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 20 client.16599 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad06392b80 0x55ad06394f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 4 client.16599 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.133+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad062d8620 con 0x55ad06392b80 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.134+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad06392b80 0x55ad06394f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.134+0000 7fdcadffb700 10 client.16599 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.134+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad06392b80 0x55ad06394f70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4442e085f59995 server_cookie=ccb23db2611b8f8e in_seq=0 out_seq=0 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcadffb700 1 -- 172.21.15.37:0/1004953871 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad062d8620 con 0x55ad06392b80 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcadffb700 10 client.16599 handle_command_reply: tid=0 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 client.16599 shutdown 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 msgr2=0x55ad0637fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400bf80 tx=0x7fdcb400f4c0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 shutdown_connections 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad06392b80 0x55ad06394f70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063831e0 0x55ad063801e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad063779f0 0x55ad063829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/1004953871 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad06378310 0x55ad0637fca0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 >> 172.21.15.37:0/1004953871 conn(0x55ad062a5790 msgr2=0x55ad06373b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcadffb700 0 client.16599 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 shutdown_connections 2024-04-25T21:19:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/1004953871 wait complete. 2024-04-25T21:19:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.135+0000 7fdcd6ce8b80 20 client.16599 trim_cache size 0 max 16384 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 Processor -- start 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 -- start start 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 0x55ad0639a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad0639ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41af5d0 con 0x55ad062a5790 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b1110 con 0x55ad063779f0 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41afca0 con 0x55ad06378310 2024-04-25T21:19:00.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcca1fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcca1fb700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39284/0 (socket says 172.21.15.37:39284) 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.155+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2969394483 learned_addr learned my addr 172.21.15.37:0/2969394483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 0x55ad0639a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2969394483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 msgr2=0x55ad0639ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad0639ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2969394483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 msgr2=0x55ad0639a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 0x55ad0639a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 -- 172.21.15.37:0/2969394483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad06379820 con 0x55ad062a5790 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 0x55ad0639a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006fd0 tx=0x7fdcc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115be6118b4bc904 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcaffff700 1 -- 172.21.15.37:0/2969394483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc002f070 con 0x55ad062a5790 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcaffff700 1 -- 172.21.15.37:0/2969394483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcc0009e40 con 0x55ad062a5790 2024-04-25T21:19:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.156+0000 7fdcaffff700 1 -- 172.21.15.37:0/2969394483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0024040 con 0x55ad062a5790 2024-04-25T21:19:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2969394483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 msgr2=0x55ad06381be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0006fd0 tx=0x7fdcc00091a0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2969394483 shutdown_connections 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad0639ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad062a5790 0x55ad06381be0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/2969394483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad063779f0 0x55ad0639a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2969394483 >> 172.21.15.37:0/2969394483 conn(0x55ad06374110 msgr2=0x55ad063814d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2969394483 shutdown_connections 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/2969394483 wait complete. 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- start start 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 Processor -- start 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad062a5790 0x55ad0638b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06389aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b0a40 con 0x55ad063779f0 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b0370 con 0x55ad062a5790 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ad590 con 0x55ad06378310 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.157+0000 7fdcd6ce8b80 10 client.0 _ll_register_callbacks cb 0x55ad06263a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39290/0 (socket says 172.21.15.37:39290) 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 -- 172.21.15.37:0/62164988 learned_addr learned my addr 172.21.15.37:0/62164988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad062a5790 0x55ad0638b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcca9fc700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06389aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 -- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 msgr2=0x55ad06389aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06389aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 -- 172.21.15.37:0/62164988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad062a5790 msgr2=0x55ad0638b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad062a5790 0x55ad0638b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 -- 172.21.15.37:0/62164988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad0638a160 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcc99fa700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bcb999d38aec8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcadffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb402b070 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fdcb40135c0 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcc0009e40 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcb4037050 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcd6ce8b80 10 client.16605 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.158+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad063f9fc0 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcb40137e0 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdcb4030440 con 0x55ad063779f0 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 10 client.16605 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 20 client.16605 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad063831e0 0x55ad063fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 4 client.16605 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "308523c7", "release": true, "await": true}] 2024-04-25T21:19:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "308523c7", "release": true, "await": true}) -- 0x7fdcc0009e40 con 0x55ad063831e0 2024-04-25T21:19:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad063831e0 0x55ad063fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcca1fb700 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad063831e0 0x55ad063fc6b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1f01a58cce5ec70 server_cookie=d65841451eefad2 in_seq=0 out_seq=0 2024-04-25T21:19:00.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.159+0000 7fdcadffb700 10 client.16605 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.163+0000 7fdcadffb700 1 -- 172.21.15.37:0/62164988 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fdcc0009e40 con 0x55ad063831e0 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.163+0000 7fdcadffb700 10 client.16605 handle_command_reply: tid=0 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.163+0000 7fdcd6ce8b80 1 client.16605 shutdown 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.163+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 msgr2=0x55ad06389560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4004750 comp rx=0 tx=0).stop 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 shutdown_connections 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ad063831e0 0x55ad063fc6b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ad06378310 0x55ad06389aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ad062a5790 0x55ad0638b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcadffb700 0 client.16605 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 --2- 172.21.15.37:0/62164988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad063779f0 0x55ad06389560 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 >> 172.21.15.37:0/62164988 conn(0x55ad06374110 msgr2=0x55ad06381d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 shutdown_connections 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 1 -- 172.21.15.37:0/62164988 wait complete. 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdcd6ce8b80 20 client.16605 trim_cache size 0 max 16384 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 28, 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "308523c7": { 2024-04-25T21:19:00.166 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 28, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.3, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:19:00.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:00.168 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 msgr2=0x7fdcb00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 0x7fdcb00a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00099d0 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 msgr2=0x7fdcc419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b070 tx=0x7fdcbc00cf00 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 shutdown_connections 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdcb00a3db0 0x7fdcb00a6270 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcc4108a50 0x7fdcc419fdf0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcc4102a90 0x7fdcc419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 --2- 172.21.15.37:0/1920451909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdcc4103450 0x7fdcc419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:00.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 >> 172.21.15.37:0/1920451909 conn(0x7fdcc4079ba0 msgr2=0x7fdcc410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.164+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 shutdown_connections 2024-04-25T21:19:00.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:00.165+0000 7fdccc45f700 1 -- 172.21.15.37:0/1920451909 wait complete. 2024-04-25T21:19:00.846 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '308523c7', seconds elapsed: 36.1 2024-04-25T21:19:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 -- 172.21.15.37:0/3525549625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e4101a60 msgr2=0x7f55e4105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/3525549625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e4101a60 0x7f55e4105aa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009900 tx=0x7f55d402a410 comp rx=0 tx=0).stop 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 -- 172.21.15.37:0/3525549625 shutdown_connections 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/3525549625 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e4106180 0x7f55e410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/3525549625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e4101a60 0x7f55e4105aa0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/3525549625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e41010b0 0x7f55e4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 -- 172.21.15.37:0/3525549625 >> 172.21.15.37:0/3525549625 conn(0x7f55e4078000 msgr2=0x7f55e4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 -- 172.21.15.37:0/3525549625 shutdown_connections 2024-04-25T21:19:01.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.793+0000 7f55ebfcc640 1 -- 172.21.15.37:0/3525549625 wait complete. 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 Processor -- start 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 -- start start 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e41010b0 0x7f55e41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 0x7f55e4198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41121c0 con 0x7f55e419cf50 2024-04-25T21:19:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4112040 con 0x7f55e4106180 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ebfcc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4112340 con 0x7f55e41010b0 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ea569640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ea569640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39298/0 (socket says 172.21.15.37:39298) 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.794+0000 7f55ea569640 1 -- 172.21.15.37:0/4249062133 learned_addr learned my addr 172.21.15.37:0/4249062133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55e9d68640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e41010b0 0x7f55e41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55e9567640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 0x7f55e4198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 -- 172.21.15.37:0/4249062133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e41010b0 msgr2=0x7f55e41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e41010b0 0x7f55e41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 -- 172.21.15.37:0/4249062133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 msgr2=0x7f55e4198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 0x7f55e4198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 -- 172.21.15.37:0/4249062133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e419da40 con 0x7f55e419cf50 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55e9567640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 0x7f55e4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ea569640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00eaa0 tx=0x7f55dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55daffd640 1 -- 172.21.15.37:0/4249062133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00cdb0 con 0x7f55e419cf50 2024-04-25T21:19:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55daffd640 1 -- 172.21.15.37:0/4249062133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55dc005020 con 0x7f55e419cf50 2024-04-25T21:19:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e419dd30 con 0x7f55e419cf50 2024-04-25T21:19:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55daffd640 1 -- 172.21.15.37:0/4249062133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc0116d0 con 0x7f55e419cf50 2024-04-25T21:19:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.795+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e41a54f0 con 0x7f55e419cf50 2024-04-25T21:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55daffd640 1 -- 172.21.15.37:0/4249062133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55dc0051d0 con 0x7f55e419cf50 2024-04-25T21:19:01.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 Processor -- start 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 -- start start 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01cff620 0x55ea01cffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ea01c64970 con 0x55ea01d0a800 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ea01c014a0 con 0x55ea01cff620 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.798+0000 7f55eedb6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea01c60930 con 0x55ea01cfff40 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39310/0 (socket says 172.21.15.37:39310) 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 -- 172.21.15.37:0/1663096387 learned_addr learned my addr 172.21.15.37:0/1663096387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55daffd640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 0x7f55d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01cff620 0x55ea01cffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9d68640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55ea569640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 0x7f55d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 -- 172.21.15.37:0/1663096387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 msgr2=0x55ea01d0a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d0a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 -- 172.21.15.37:0/1663096387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01cff620 msgr2=0x55ea01cffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01cff620 0x55ea01cffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 -- 172.21.15.37:0/1663096387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea01977210 con 0x55ea01d0a800 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9d68640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d0a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55daffd640 1 -- 172.21.15.37:0/4249062133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f55dc0d08f0 con 0x7f55e419cf50 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55e9567640 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f55e406a620 tx=0x7f55d4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd37d21bc2aae5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55ea569640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 0x7f55d00a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ce0f0 tx=0x7f55dc06f380 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.799+0000 7f55cffff640 1 -- 172.21.15.37:0/1663096387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4035040 con 0x55ea01d0a800 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55cffff640 1 -- 172.21.15.37:0/1663096387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40035f0 con 0x55ea01d0a800 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55cffff640 1 -- 172.21.15.37:0/1663096387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d40044e0 con 0x55ea01d0a800 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 -- 172.21.15.37:0/1663096387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 msgr2=0x55ea01d0cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f55e406a620 tx=0x7f55d4005b50 comp rx=0 tx=0).stop 2024-04-25T21:19:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 -- 172.21.15.37:0/1663096387 shutdown_connections 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01d0a800 0x55ea01d0cbf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d0a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1663096387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01cff620 0x55ea01cffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 -- 172.21.15.37:0/1663096387 >> 172.21.15.37:0/1663096387 conn(0x55ea01cfbb30 msgr2=0x55ea01cfbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 -- 172.21.15.37:0/1663096387 shutdown_connections 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.800+0000 7f55eedb6740 1 -- 172.21.15.37:0/1663096387 wait complete. 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 -- start start 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 Processor -- start 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ea01d0d3d0 con 0x55ea01cff620 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ea01bff0b0 con 0x55ea01d0a800 2024-04-25T21:19:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ea01d0d550 con 0x55ea01cfff40 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55eedb6740 10 client.0 _ll_register_callbacks cb 0x55ea01becd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39316/0 (socket says 172.21.15.37:39316) 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.801+0000 7f55e9567640 1 -- 172.21.15.37:0/1780951206 learned_addr learned my addr 172.21.15.37:0/1780951206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9d68640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 -- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 msgr2=0x55ea01d07460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d07460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 -- 172.21.15.37:0/1780951206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 msgr2=0x55ea01d079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 -- 172.21.15.37:0/1780951206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea01a43620 con 0x55ea01cff620 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55e9567640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005ec0 tx=0x7f55d4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac7768aef54218c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4000e60 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f55d4031b10 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d40035f0 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55d4035070 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55eedb6740 10 client.16617 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.802+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea01d19300 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d40035f0 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f55d403a040 con 0x55ea01cff620 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 10 client.16617 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 20 client.16617 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d1a650 0x55ea01d1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 4 client.16617 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea01c72390 con 0x55ea01d1a650 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55e9d68640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d1a650 0x55ea01d1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55cd7fa640 10 client.16617 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.803+0000 7f55e9d68640 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d1a650 0x55ea01d1ca40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116950ceeccb9c59 server_cookie=64ed6e13abf77be8 in_seq=0 out_seq=0 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55cd7fa640 1 -- 172.21.15.37:0/1780951206 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea01c72390 con 0x55ea01d1a650 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55cd7fa640 10 client.16617 handle_command_reply: tid=0 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 client.16617 shutdown 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 msgr2=0x55ea01d0a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005ec0 tx=0x7f55d4003ed0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 shutdown_connections 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d1a650 0x55ea01d1ca40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d07460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 --2- 172.21.15.37:0/1780951206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d0a170 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55cd7fa640 0 client.16617 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.804+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 >> 172.21.15.37:0/1780951206 conn(0x55ea01cfbb30 msgr2=0x55ea01d01860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.805+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 shutdown_connections 2024-04-25T21:19:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.805+0000 7f55eedb6740 1 -- 172.21.15.37:0/1780951206 wait complete. 2024-04-25T21:19:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.805+0000 7f55eedb6740 20 client.16617 trim_cache size 0 max 16384 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 Processor -- start 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 -- start start 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41adae0 con 0x55ea01cff620 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41af620 con 0x55ea01d0a800 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55eedb6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41ae1b0 con 0x55ea01cfff40 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39326/0 (socket says 172.21.15.37:39326) 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.820+0000 7f55e9567640 1 -- 172.21.15.37:0/2673162921 learned_addr learned my addr 172.21.15.37:0/2673162921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9d68640 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 -- 172.21.15.37:0/2673162921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 msgr2=0x55ea01d1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 -- 172.21.15.37:0/2673162921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 msgr2=0x55ea01d1ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 -- 172.21.15.37:0/2673162921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea01d06410 con 0x55ea01cff620 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9d68640 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55e9567640 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f55d40059d0 tx=0x7f55d40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57892ce4ffa18995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55cf7fe640 1 -- 172.21.15.37:0/2673162921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4009780 con 0x55ea01cff620 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55cf7fe640 1 -- 172.21.15.37:0/2673162921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d4041030 con 0x55ea01cff620 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55cf7fe640 1 -- 172.21.15.37:0/2673162921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4031a80 con 0x55ea01cff620 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55eedb6740 1 -- 172.21.15.37:0/2673162921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 msgr2=0x55ea01d15eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.821+0000 7f55eedb6740 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f55d40059d0 tx=0x7f55d40056d0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- 172.21.15.37:0/2673162921 shutdown_connections 2024-04-25T21:19:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- 172.21.15.37:0/2673162921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d15eb0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- 172.21.15.37:0/2673162921 >> 172.21.15.37:0/2673162921 conn(0x55ea01cfbb30 msgr2=0x55ea01d157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- 172.21.15.37:0/2673162921 shutdown_connections 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- 172.21.15.37:0/2673162921 wait complete. 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- start start 2024-04-25T21:19:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 Processor -- start 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d7db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41aef50 con 0x55ea01cff620 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41ae880 con 0x55ea01d0a800 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41abae0 con 0x55ea01cfff40 2024-04-25T21:19:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.822+0000 7f55eedb6740 10 client.0 _ll_register_callbacks cb 0x7f55d40316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39338/0 (socket says 172.21.15.37:39338) 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 -- 172.21.15.37:0/4059522801 learned_addr learned my addr 172.21.15.37:0/4059522801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9d68640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d7db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 -- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 msgr2=0x55ea01d7db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d7db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 -- 172.21.15.37:0/4059522801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 msgr2=0x55ea01d1f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 -- 172.21.15.37:0/4059522801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea01cffd70 con 0x55ea01cff620 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9d68640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55e9567640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f55d4032ec0 tx=0x7f55d4031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fc7c52229c8ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4009780 con 0x55ea01cff620 2024-04-25T21:19:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d4041030 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f55d4003b10 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f55d4003d10 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55eedb6740 10 client.16623 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.823+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ea01d07d30 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d4041030 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f55d4060430 con 0x55ea01cff620 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 10 client.16623 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 20 client.16623 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d84e00 0x55ea01d87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 4 client.16623 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea01c6c200 con 0x55ea01d84e00 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.824+0000 7f55e9d68640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d84e00 0x55ea01d87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.825+0000 7f55e9d68640 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d84e00 0x55ea01d87220 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260d6292d950202d server_cookie=1135fef41b12e1d6 in_seq=0 out_seq=0 2024-04-25T21:19:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.825+0000 7f55cd7fa640 10 client.16623 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.836+0000 7f55cd7fa640 1 -- 172.21.15.37:0/4059522801 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea01c6c200 con 0x55ea01d84e00 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.836+0000 7f55cd7fa640 10 client.16623 handle_command_reply: tid=0 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.836+0000 7f55eedb6740 1 client.16623 shutdown 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 msgr2=0x55ea01d7d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f55d4032ec0 tx=0x7f55d4031510 comp rx=0 tx=0).stop 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 shutdown_connections 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ea01d84e00 0x55ea01d87220 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ea01d0a800 0x55ea01d1f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea01cfff40 0x55ea01d7db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55cd7fa640 0 client.16623 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 --2- 172.21.15.37:0/4059522801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ea01cff620 0x55ea01d7d5f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 >> 172.21.15.37:0/4059522801 conn(0x55ea01cfbb30 msgr2=0x55ea01d16030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 shutdown_connections 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 1 -- 172.21.15.37:0/4059522801 wait complete. 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.837+0000 7f55eedb6740 20 client.16623 trim_cache size 0 max 16384 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:19:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 msgr2=0x7f55d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 0x7f55d00a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0ce0f0 tx=0x7f55dc06f380 comp rx=0 tx=0).stop 2024-04-25T21:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 msgr2=0x7f55e419d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00eaa0 tx=0x7f55dc00ef60 comp rx=0 tx=0).stop 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 shutdown_connections 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55e419cf50 0x7f55e419d400 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55e4106180 0x7f55e4198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f55d00a3e40 0x7f55d00a6300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 --2- 172.21.15.37:0/4249062133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55e41010b0 0x7f55e41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 >> 172.21.15.37:0/4249062133 conn(0x7f55e4078000 msgr2=0x7f55e40ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 shutdown_connections 2024-04-25T21:19:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:01.838+0000 7f55ebfcc640 1 -- 172.21.15.37:0/4249062133 wait complete. 2024-04-25T21:19:01.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:01 smithi177 ceph-mon[33043]: pgmap v327: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:19:01.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:01 smithi098 ceph-mon[32569]: pgmap v327: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:19:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:01 smithi037 ceph-mon[29195]: pgmap v327: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-25T21:19:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:03 smithi177 ceph-mon[33043]: pgmap v328: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:19:03.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:03 smithi098 ceph-mon[32569]: pgmap v328: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:19:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:03 smithi037 ceph-mon[29195]: pgmap v328: 97 pgs: 97 active+clean; 1.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:19:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:05 smithi177 ceph-mon[33043]: pgmap v329: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-25T21:19:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:05 smithi098 ceph-mon[32569]: pgmap v329: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-25T21:19:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:05 smithi037 ceph-mon[29195]: pgmap v329: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-25T21:19:07.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:07 smithi037 ceph-mon[29195]: pgmap v330: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 29 op/s 2024-04-25T21:19:07.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:07 smithi037 ceph-mon[29195]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:19:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:07 smithi177 ceph-mon[33043]: pgmap v330: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 29 op/s 2024-04-25T21:19:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:07 smithi177 ceph-mon[33043]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:19:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:07 smithi098 ceph-mon[32569]: pgmap v330: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 64 KiB/s wr, 29 op/s 2024-04-25T21:19:07.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:07 smithi098 ceph-mon[32569]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:19:09.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:09 smithi177 ceph-mon[33043]: pgmap v332: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-04-25T21:19:09.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:09 smithi098 ceph-mon[32569]: pgmap v332: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-04-25T21:19:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:09 smithi037 ceph-mon[29195]: pgmap v332: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 76 KiB/s wr, 35 op/s 2024-04-25T21:19:11.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:11 smithi177 ceph-mon[33043]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:11.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:11 smithi098 ceph-mon[32569]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:11 smithi037 ceph-mon[29195]: pgmap v333: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:13.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:13 smithi177 ceph-mon[33043]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:13.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:13 smithi098 ceph-mon[32569]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:13 smithi037 ceph-mon[29195]: pgmap v334: 97 pgs: 97 active+clean; 1.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.939+0000 7f1b1057d640 1 -- 172.21.15.37:0/2056575974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08102b70 msgr2=0x7f1b08106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.939+0000 7f1b1057d640 1 --2- 172.21.15.37:0/2056575974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08102b70 0x7f1b08106bb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009900 tx=0x7f1af802a410 comp rx=0 tx=0).stop 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- 172.21.15.37:0/2056575974 shutdown_connections 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- 172.21.15.37:0/2056575974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b08107290 0x7f1b0810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- 172.21.15.37:0/2056575974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08102b70 0x7f1b08106bb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- 172.21.15.37:0/2056575974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b081021c0 0x7f1b081025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- 172.21.15.37:0/2056575974 >> 172.21.15.37:0/2056575974 conn(0x7f1b080fda20 msgr2=0x7f1b080ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- 172.21.15.37:0/2056575974 shutdown_connections 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- 172.21.15.37:0/2056575974 wait complete. 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 Processor -- start 2024-04-25T21:19:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- start start 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b081021c0 0x7f1b08074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b08074cd0 0x7f1b081a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081121b0 con 0x7f1b08107290 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08112030 con 0x7f1b08074cd0 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.940+0000 7f1b1057d640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08112330 con 0x7f1b081021c0 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49442/0 (socket says 172.21.15.37:49442) 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 -- 172.21.15.37:0/255023240 learned_addr learned my addr 172.21.15.37:0/255023240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0e319640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b081021c0 0x7f1b08074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b08074cd0 0x7f1b081a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 -- 172.21.15.37:0/255023240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b081021c0 msgr2=0x7f1b08074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b081021c0 0x7f1b08074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 -- 172.21.15.37:0/255023240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b08074cd0 msgr2=0x7f1b081a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b08074cd0 0x7f1b081a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 -- 172.21.15.37:0/255023240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b081a6f60 con 0x7f1b08107290 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b0db18640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1af8006010 tx=0x7f1af8004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1aff7fe640 1 -- 172.21.15.37:0/255023240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af803f030 con 0x7f1b08107290 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1aff7fe640 1 -- 172.21.15.37:0/255023240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af8003aa0 con 0x7f1b08107290 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b081a71f0 con 0x7f1b08107290 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1aff7fe640 1 -- 172.21.15.37:0/255023240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8035070 con 0x7f1b08107290 2024-04-25T21:19:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.941+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b081a7730 con 0x7f1b08107290 2024-04-25T21:19:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.944+0000 7f1aff7fe640 1 -- 172.21.15.37:0/255023240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1af8042030 con 0x7f1b08107290 2024-04-25T21:19:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.944+0000 7f1b13373740 1 Processor -- start 2024-04-25T21:19:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.944+0000 7f1b13373740 1 -- start start 2024-04-25T21:19:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2da0a8970 con 0x55d2da143620 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d2da0454a0 con 0x55d2da14e800 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b13373740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d2da0a4930 con 0x55d2da143f40 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1aff7fe640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 0x7f1af40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49450/0 (socket says 172.21.15.37:49450) 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 -- 172.21.15.37:0/3094611799 learned_addr learned my addr 172.21.15.37:0/3094611799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 0x7f1af40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0db18640 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0e319640 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1aff7fe640 1 -- 172.21.15.37:0/255023240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f1af80f0ba0 con 0x7f1b08107290 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 -- 172.21.15.37:0/3094611799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 msgr2=0x55d2da14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 -- 172.21.15.37:0/3094611799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 msgr2=0x55d2da150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.945+0000 7f1b0eb1a640 1 -- 172.21.15.37:0/3094611799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d9dbb210 con 0x55d2da143620 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.946+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1b000094d0 tx=0x7f1b000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42da2a8a76d676b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.946+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 0x7f1af40a6330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1b00005c40 tx=0x7f1b0001d000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.946+0000 7f1afcbfe640 1 -- 172.21.15.37:0/3094611799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b00012840 con 0x55d2da143620 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.946+0000 7f1afcbfe640 1 -- 172.21.15.37:0/3094611799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b00009d60 con 0x55d2da143620 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.946+0000 7f1afcbfe640 1 -- 172.21.15.37:0/3094611799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000de00 con 0x55d2da143620 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- 172.21.15.37:0/3094611799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 msgr2=0x55d2da143a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1b000094d0 tx=0x7f1b000107a0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- 172.21.15.37:0/3094611799 shutdown_connections 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- 172.21.15.37:0/3094611799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da143a00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- 172.21.15.37:0/3094611799 >> 172.21.15.37:0/3094611799 conn(0x55d2da13fb30 msgr2=0x55d2da13ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- 172.21.15.37:0/3094611799 shutdown_connections 2024-04-25T21:19:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- 172.21.15.37:0/3094611799 wait complete. 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- start start 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 Processor -- start 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143620 0x55d2da14e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.947+0000 7f1b13373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2da1513d0 con 0x55d2da143f40 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b13373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d2da0430b0 con 0x55d2da14e800 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b13373740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d2da151550 con 0x55d2da143620 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b13373740 10 client.0 _ll_register_callbacks cb 0x55d2da030d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49458/0 (socket says 172.21.15.37:49458) 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 -- 172.21.15.37:0/826099474 learned_addr learned my addr 172.21.15.37:0/826099474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0e319640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143620 0x55d2da14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 -- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143620 msgr2=0x55d2da14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143620 0x55d2da14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 -- 172.21.15.37:0/826099474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 msgr2=0x55d2da14bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 -- 172.21.15.37:0/826099474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d9e87620 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b0db18640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1af80b8f80 tx=0x7f1af802ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e808d477975c44a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8049040 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f1af8003c50 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2da0b6390 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1af80bc480 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.948+0000 7f1b13373740 10 client.16635 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2da15d3c0 con 0x55d2da143f40 2024-04-25T21:19:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d2da0b6390 con 0x55d2da143f40 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1af80dbce0 con 0x55d2da143f40 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 10 client.16635 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 20 client.16635 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da15e7d0 0x55d2da160c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 4 client.16635 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2da0b0200 con 0x55d2da15e7d0 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.949+0000 7f1b0e319640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da15e7d0 0x55d2da160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.950+0000 7f1af1ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.950+0000 7f1b0e319640 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da15e7d0 0x55d2da160c10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36d76b7d96470654 server_cookie=9bda14e325bcdfac in_seq=0 out_seq=0 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.950+0000 7f1af1ffb640 1 -- 172.21.15.37:0/826099474 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2da0b0200 con 0x55d2da15e7d0 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.950+0000 7f1af1ffb640 10 client.16635 handle_command_reply: tid=0 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.950+0000 7f1b13373740 1 client.16635 shutdown 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 msgr2=0x55d2da14b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1af80b8f80 tx=0x7f1af802ffa0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 shutdown_connections 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da15e7d0 0x55d2da160c10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143f40 0x55d2da14b500 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 --2- 172.21.15.37:0/826099474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143620 0x55d2da14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1af1ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 >> 172.21.15.37:0/826099474 conn(0x55d2da13fb30 msgr2=0x55d2da145860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 shutdown_connections 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 1 -- 172.21.15.37:0/826099474 wait complete. 2024-04-25T21:19:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.951+0000 7f1b13373740 20 client.16635 trim_cache size 0 max 16384 2024-04-25T21:19:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 Processor -- start 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 -- start start 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081afd30 con 0x55d2da143620 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081b1870 con 0x55d2da14e800 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b13373740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081b0400 con 0x55d2da143f40 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0e319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0e319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49474/0 (socket says 172.21.15.37:49474) 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0e319640 1 -- 172.21.15.37:0/1286595161 learned_addr learned my addr 172.21.15.37:0/1286595161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0e319640 1 -- 172.21.15.37:0/1286595161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 msgr2=0x55d2da13eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0db18640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.966+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da13eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0e319640 1 -- 172.21.15.37:0/1286595161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 msgr2=0x55d2da14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0e319640 1 -- 172.21.15.37:0/1286595161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2da161290 con 0x55d2da143620 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0db18640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b0400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e777e80545b271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1af3fff640 1 -- 172.21.15.37:0/1286595161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004220 con 0x55d2da143620 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1af3fff640 1 -- 172.21.15.37:0/1286595161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b04024030 con 0x55d2da143620 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1af3fff640 1 -- 172.21.15.37:0/1286595161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04004d30 con 0x55d2da143620 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 -- 172.21.15.37:0/1286595161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 msgr2=0x55d2da15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1b04000c00 tx=0x7f1b0400b410 comp rx=0 tx=0).stop 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 -- 172.21.15.37:0/1286595161 shutdown_connections 2024-04-25T21:19:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da13eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 --2- 172.21.15.37:0/1286595161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da15a290 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.967+0000 7f1b13373740 1 -- 172.21.15.37:0/1286595161 >> 172.21.15.37:0/1286595161 conn(0x55d2da13fb30 msgr2=0x55d2da13e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- 172.21.15.37:0/1286595161 shutdown_connections 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- 172.21.15.37:0/1286595161 wait complete. 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- start start 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 Processor -- start 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da1c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081b11a0 con 0x55d2da143620 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081b0ad0 con 0x55d2da14e800 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1b081add30 con 0x55d2da143f40 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b13373740 10 client.0 _ll_register_callbacks cb 0x7f1af8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b0e319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b0e319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49490/0 (socket says 172.21.15.37:49490) 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b0e319640 1 -- 172.21.15.37:0/1434220581 learned_addr learned my addr 172.21.15.37:0/1434220581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.968+0000 7f1b0db18640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da1c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 -- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 msgr2=0x55d2da148ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da148ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 -- 172.21.15.37:0/1434220581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 msgr2=0x55d2da1c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da1c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 -- 172.21.15.37:0/1434220581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2da1c42d0 con 0x55d2da143620 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0eb1a640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da1c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b0e319640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400bca0 tx=0x7f1b040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42bc5199a865f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b040042a0 con 0x55d2da143620 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8557+0+0 (secure 0 0 0) 0x7f1b04004440 con 0x55d2da143620 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b04024030 con 0x55d2da143620 2024-04-25T21:19:14.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1b04036050 con 0x55d2da143620 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b13373740 10 client.16641 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2da1c53b0 con 0x55d2da143620 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.969+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b04024030 con 0x55d2da143620 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1b04003bc0 con 0x55d2da143620 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 10 client.16641 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 20 client.16641 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da1c5a90 0x55d2da1c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 4 client.16641 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2da02c5f0 con 0x55d2da1c5a90 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b0db18640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da1c5a90 0x55d2da1c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1af1ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:14.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.970+0000 7f1b0db18640 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da1c5a90 0x55d2da1c7eb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34c0b50f5c5bfe64 server_cookie=a8cb1fb73bf48a3a in_seq=0 out_seq=0 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.984+0000 7f1af1ffb640 1 -- 172.21.15.37:0/1434220581 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2da02c5f0 con 0x55d2da1c5a90 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.984+0000 7f1af1ffb640 10 client.16641 handle_command_reply: tid=0 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.984+0000 7f1b13373740 1 client.16641 shutdown 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 msgr2=0x55d2da148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400bca0 tx=0x7f1b040047c0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 shutdown_connections 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2da1c5a90 0x55d2da1c7eb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d2da14e800 0x55d2da1c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1af1ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d2da143f40 0x55d2da148ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 --2- 172.21.15.37:0/1434220581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2da143620 0x55d2da148570 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 >> 172.21.15.37:0/1434220581 conn(0x55d2da13fb30 msgr2=0x55d2da145860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 shutdown_connections 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 1 -- 172.21.15.37:0/1434220581 wait complete. 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b13373740 20 client.16641 trim_cache size 0 max 16384 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:19:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 msgr2=0x7f1af40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 0x7f1af40a6330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1b00005c40 tx=0x7f1b0001d000 comp rx=0 tx=0).stop 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 msgr2=0x7f1b08074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1af8006010 tx=0x7f1af8004060 comp rx=0 tx=0).stop 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 shutdown_connections 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b08074cd0 0x7f1b081a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b08107290 0x7f1b08074790 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1af40a3e70 0x7f1af40a6330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 --2- 172.21.15.37:0/255023240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b081021c0 0x7f1b08074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.985+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 >> 172.21.15.37:0/255023240 conn(0x7f1b080fda20 msgr2=0x7f1b080ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.986+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 shutdown_connections 2024-04-25T21:19:14.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:14.986+0000 7f1b1057d640 1 -- 172.21.15.37:0/255023240 wait complete. 2024-04-25T21:19:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:15 smithi177 ceph-mon[33043]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:15.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:15 smithi177 ceph-mon[33043]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:19:15.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:15 smithi098 ceph-mon[32569]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:15 smithi098 ceph-mon[32569]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:19:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:15 smithi037 ceph-mon[29195]: pgmap v335: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:15 smithi037 ceph-mon[29195]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:19:17.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:17 smithi098 ceph-mon[32569]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:17 smithi037 ceph-mon[29195]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:17 smithi177 ceph-mon[33043]: pgmap v337: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:19.891 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:19 smithi177 ceph-mon[33043]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:19.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:19 smithi098 ceph-mon[32569]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:19 smithi037 ceph-mon[29195]: pgmap v338: 97 pgs: 97 active+clean; 2.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:21.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:21 smithi098 ceph-mon[32569]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:21 smithi037 ceph-mon[29195]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:21 smithi177 ceph-mon[33043]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:23.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:23 smithi098 ceph-mon[32569]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:23 smithi037 ceph-mon[29195]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:23 smithi177 ceph-mon[33043]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:19:25.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:25 smithi098 ceph-mon[32569]: pgmap v341: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:19:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:25 smithi037 ceph-mon[29195]: pgmap v341: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:19:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:25 smithi177 ceph-mon[33043]: pgmap v341: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:19:27.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:27 smithi098 ceph-mon[32569]: pgmap v342: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:19:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:27 smithi037 ceph-mon[29195]: pgmap v342: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/4240066515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 msgr2=0x7fe1d810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/4240066515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d810ea00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009ca0 tx=0x7fe1d002a950 comp rx=0 tx=0).stop 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/4240066515 shutdown_connections 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/4240066515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d810ea00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/4240066515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103b20 0x7fe1d810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/4240066515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103200 0x7fe1d81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/4240066515 >> 172.21.15.37:0/4240066515 conn(0x7fe1d80771d0 msgr2=0x7fe1d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/4240066515 shutdown_connections 2024-04-25T21:19:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.084+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/4240066515 wait complete. 2024-04-25T21:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.085+0000 7fe1dcf0f640 1 Processor -- start 2024-04-25T21:19:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 -- start start 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103200 0x7fe1d8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103b20 0x7fe1d8198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81121f0 con 0x7fe1d810c5d0 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8112070 con 0x7fe1d8103b20 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1dcf0f640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8112370 con 0x7fe1d8103200 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38384/0 (socket says 172.21.15.37:38384) 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 -- 172.21.15.37:0/274196773 learned_addr learned my addr 172.21.15.37:0/274196773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103b20 0x7fe1d8198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d659c640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103200 0x7fe1d8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 -- 172.21.15.37:0/274196773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103200 msgr2=0x7fe1d8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103200 0x7fe1d8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 -- 172.21.15.37:0/274196773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103b20 msgr2=0x7fe1d8198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103b20 0x7fe1d8198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.086+0000 7fe1d6d9d640 1 -- 172.21.15.37:0/274196773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d819d610 con 0x7fe1d810c5d0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1d6d9d640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe1d002ae70 tx=0x7fe1d002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1cffff640 1 -- 172.21.15.37:0/274196773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0034040 con 0x7fe1d810c5d0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1cffff640 1 -- 172.21.15.37:0/274196773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d00308f0 con 0x7fe1d810c5d0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d819d8a0 con 0x7fe1d810c5d0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1cffff640 1 -- 172.21.15.37:0/274196773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00047c0 con 0x7fe1d810c5d0 2024-04-25T21:19:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.087+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d81a5190 con 0x7fe1d810c5d0 2024-04-25T21:19:28.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:27 smithi177 ceph-mon[33043]: pgmap v342: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 45 KiB/s wr, 17 op/s 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.089+0000 7fe1cffff640 1 -- 172.21.15.37:0/274196773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1d0037020 con 0x7fe1d810c5d0 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.089+0000 7fe1dfc22740 1 Processor -- start 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 -- start start 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cb1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cbc800 0x5583a7cbebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5583a7c16970 con 0x5583a7cb1f40 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583a7bb34a0 con 0x5583a7cbc800 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5583a7c12930 con 0x5583a7cb1620 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38400/0 (socket says 172.21.15.37:38400) 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1d659c640 1 -- 172.21.15.37:0/1060780053 learned_addr learned my addr 172.21.15.37:0/1060780053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1cffff640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 0x7fe1c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cb1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cbc800 0x5583a7cbebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.090+0000 7fe1d6d9d640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 0x7fe1c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 -- 172.21.15.37:0/1060780053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 msgr2=0x5583a7cb1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cb1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 -- 172.21.15.37:0/1060780053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cbc800 msgr2=0x5583a7cbebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cbc800 0x5583a7cbebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 -- 172.21.15.37:0/1060780053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a7929210 con 0x5583a7cb1f40 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1cffff640 1 -- 172.21.15.37:0/274196773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fe1d00f1610 con 0x7fe1d810c5d0 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d659c640 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800bd10 tx=0x7fe1c80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5579b83d7e78188d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1d6d9d640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 0x7fe1c40a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00eef20 tx=0x7fe1d008f100 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1ccd99640 1 -- 172.21.15.37:0/1060780053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8004030 con 0x5583a7cb1f40 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1ccd99640 1 -- 172.21.15.37:0/1060780053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c80035f0 con 0x5583a7cb1f40 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.091+0000 7fe1ccd99640 1 -- 172.21.15.37:0/1060780053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8011770 con 0x5583a7cb1f40 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- 172.21.15.37:0/1060780053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 msgr2=0x5583a7cbc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800bd10 tx=0x7fe1c80107a0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- 172.21.15.37:0/1060780053 shutdown_connections 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cbc800 0x5583a7cbebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cbc2c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/1060780053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cb1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- 172.21.15.37:0/1060780053 >> 172.21.15.37:0/1060780053 conn(0x5583a7cadb30 msgr2=0x5583a7cadf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- 172.21.15.37:0/1060780053 shutdown_connections 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- 172.21.15.37:0/1060780053 wait complete. 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 -- start start 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.092+0000 7fe1dfc22740 1 Processor -- start 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cbc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cb9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5583a7cbf430 con 0x5583a7cbc800 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5583a7bb10b0 con 0x5583a7cb1f40 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5583a7cbf5b0 con 0x5583a7cb1620 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1dfc22740 10 client.0 _ll_register_callbacks cb 0x5583a7b9ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38408/0 (socket says 172.21.15.37:38408) 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/422248734 learned_addr learned my addr 172.21.15.37:0/422248734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:28.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cbc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1d659c640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cb9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 msgr2=0x5583a7cbc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cbc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/422248734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 msgr2=0x5583a7cb9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cb9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/422248734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a79f5620 con 0x5583a7cbc800 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1d659c640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cb9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0013b50 tx=0x7fe1c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcdcdc36dcaf26c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.093+0000 7fe1bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0013c40 con 0x5583a7cbc800 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fe1c0013de0 con 0x5583a7cbc800 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c80035f0 con 0x5583a7cbc800 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1c000b4a0 con 0x5583a7cbc800 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 10 client.16653 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583a7ccb3b0 con 0x5583a7cbc800 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c80035f0 con 0x5583a7cbc800 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c00047a0 con 0x5583a7cbc800 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 10 client.16653 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 20 client.16653 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7ccc820 0x5583a7ccec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 4 client.16653 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.094+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a7c24390 con 0x5583a7ccc820 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.095+0000 7fe1d659c640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7ccc820 0x5583a7ccec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.095+0000 7fe1d659c640 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7ccc820 0x5583a7ccec10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f249730ba05e7c server_cookie=438d428fcfe5f7de in_seq=0 out_seq=0 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.095+0000 7fe1bdffb640 10 client.16653 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1bdffb640 1 -- 172.21.15.37:0/422248734 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583a7c24390 con 0x5583a7ccc820 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1bdffb640 10 client.16653 handle_command_reply: tid=0 2024-04-25T21:19:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 client.16653 shutdown 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 msgr2=0x5583a7cb9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0013b50 tx=0x7fe1c00046c0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 shutdown_connections 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7ccc820 0x5583a7ccec10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cb9950 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1bdffb640 0 client.16653 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cb9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/422248734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cbc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 >> 172.21.15.37:0/422248734 conn(0x5583a7cadb30 msgr2=0x5583a7caccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 shutdown_connections 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 1 -- 172.21.15.37:0/422248734 wait complete. 2024-04-25T21:19:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.096+0000 7fe1dfc22740 20 client.16653 trim_cache size 0 max 16384 2024-04-25T21:19:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 Processor -- start 2024-04-25T21:19:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 -- start start 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cc82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cacb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81adac0 con 0x5583a7cbc800 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81af600 con 0x5583a7cb1f40 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81ae190 con 0x5583a7cb1620 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.111+0000 7fe1cdd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38416/0 (socket says 172.21.15.37:38416) 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/4080362775 learned_addr learned my addr 172.21.15.37:0/4080362775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cc82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1d659c640 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cacb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/4080362775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 msgr2=0x5583a7cc82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cc82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/4080362775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 msgr2=0x5583a7cacb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cacb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 -- 172.21.15.37:0/4080362775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a7cb7980 con 0x5583a7cbc800 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0013aa0 tx=0x7fe1c000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fefa6abfda942d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1bffff640 1 -- 172.21.15.37:0/4080362775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000b390 con 0x5583a7cbc800 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1bffff640 1 -- 172.21.15.37:0/4080362775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0031070 con 0x5583a7cbc800 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.112+0000 7fe1bffff640 1 -- 172.21.15.37:0/4080362775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004fd0 con 0x5583a7cbc800 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- 172.21.15.37:0/4080362775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 msgr2=0x5583a7cd44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0013aa0 tx=0x7fe1c000bb00 comp rx=0 tx=0).stop 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- 172.21.15.37:0/4080362775 shutdown_connections 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cbc800 0x5583a7cd44e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1f40 0x5583a7cacb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/4080362775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cb1620 0x5583a7cc82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- 172.21.15.37:0/4080362775 >> 172.21.15.37:0/4080362775 conn(0x5583a7cadb30 msgr2=0x5583a7cac420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- 172.21.15.37:0/4080362775 shutdown_connections 2024-04-25T21:19:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- 172.21.15.37:0/4080362775 wait complete. 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- start start 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 Processor -- start 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 0x5583a7cc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cbc800 0x5583a7cc1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81aef30 con 0x5583a7cb1f40 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81ae860 con 0x5583a7cb1620 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.113+0000 7fe1dfc22740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d81abac0 con 0x5583a7cbc800 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1dfc22740 10 client.0 _ll_register_callbacks cb 0x7fe1c0013970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38432/0 (socket says 172.21.15.37:38432) 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 -- 172.21.15.37:0/2693102522 learned_addr learned my addr 172.21.15.37:0/2693102522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 0x5583a7cc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cbc800 0x5583a7cc1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 -- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cbc800 msgr2=0x5583a7cc1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cbc800 0x5583a7cc1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 -- 172.21.15.37:0/2693102522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 msgr2=0x5583a7cc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 0x5583a7cc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 -- 172.21.15.37:0/2693102522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a7cc2570 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 0x5583a7cc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1d659c640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800bd10 tx=0x7fe1c8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f7fe6558df36fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8005450 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fe1c80055f0 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1c0031070 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1c8037050 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1dfc22740 10 client.16659 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.114+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583a7d36db0 con 0x5583a7cb1f40 2024-04-25T21:19:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1c0031070 con 0x5583a7cb1f40 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe1c80296e0 con 0x5583a7cb1f40 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 10 client.16659 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 20 client.16659 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7d37300 0x5583a7d39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 4 client.16659 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583a7cb9ce0 con 0x5583a7d37300 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7d37300 0x5583a7d39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1bdffb640 10 client.16659 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.115+0000 7fe1cdd9b640 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7d37300 0x5583a7d39720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d02189cfd33aac1b server_cookie=8484906c243b56b1 in_seq=0 out_seq=0 2024-04-25T21:19:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1bdffb640 1 -- 172.21.15.37:0/2693102522 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583a7cb9ce0 con 0x5583a7d37300 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1bdffb640 10 client.16659 handle_command_reply: tid=0 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 client.16659 shutdown 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 msgr2=0x5583a7cc4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe1c800bd10 tx=0x7fe1c8005960 comp rx=0 tx=0).stop 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 shutdown_connections 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5583a7d37300 0x5583a7d39720 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5583a7cbc800 0x5583a7cc1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583a7cb1f40 0x5583a7cc4d20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 --2- 172.21.15.37:0/2693102522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5583a7cb1620 0x5583a7cc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1bdffb640 0 client.16659 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.129+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 >> 172.21.15.37:0/2693102522 conn(0x5583a7cadb30 msgr2=0x5583a7caccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 shutdown_connections 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dfc22740 1 -- 172.21.15.37:0/2693102522 wait complete. 2024-04-25T21:19:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dfc22740 20 client.16659 trim_cache size 0 max 16384 2024-04-25T21:19:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:19:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:19:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:19:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 msgr2=0x7fe1c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 0x7fe1c40a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00eef20 tx=0x7fe1d008f100 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 msgr2=0x7fe1d819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe1d002ae70 tx=0x7fe1d002f5d0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 shutdown_connections 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1d810c5d0 0x7fe1d819cee0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1d8103b20 0x7fe1d8198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe1c40a3dc0 0x7fe1c40a6280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 --2- 172.21.15.37:0/274196773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1d8103200 0x7fe1d8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 >> 172.21.15.37:0/274196773 conn(0x7fe1d80771d0 msgr2=0x7fe1d80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 shutdown_connections 2024-04-25T21:19:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:28.130+0000 7fe1dcf0f640 1 -- 172.21.15.37:0/274196773 wait complete. 2024-04-25T21:19:29.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:29 smithi098 ceph-mon[32569]: pgmap v343: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:29 smithi037 ceph-mon[29195]: pgmap v343: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:29 smithi177 ceph-mon[33043]: pgmap v343: 97 pgs: 97 active+clean; 2.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:19:31.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:31 smithi098 ceph-mon[32569]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:19:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:31 smithi037 ceph-mon[29195]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:19:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:31 smithi177 ceph-mon[33043]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-25T21:19:33.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:33 smithi098 ceph-mon[32569]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:33 smithi037 ceph-mon[29195]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:33 smithi177 ceph-mon[33043]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:35 smithi098 ceph-mon[32569]: pgmap v346: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:19:35.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:35 smithi037 ceph-mon[29195]: pgmap v346: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:19:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:35 smithi177 ceph-mon[33043]: pgmap v346: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:313/600: open fd = 3 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:314/600: open fd = 3 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:315/600: open fd = 3 2024-04-25T21:19:37.488 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:316/600: open fd = 3 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:317/600: open fd = 3 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:318/600: open fd = 3 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.489 INFO:tasks.workunit.client.0.smithi037.stdout:319/600: open fd = 3 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:320/600: open fd = 3 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:321/600: open fd = 3 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:322/600: open fd = 3 2024-04-25T21:19:37.490 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:323/600: open fd = 3 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:324/600: open fd = 3 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:325/600: open fd = 3 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.491 INFO:tasks.workunit.client.0.smithi037.stdout:326/600: open fd = 3 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:327/600: open fd = 3 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:328/600: open fd = 3 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:329/600: open fd = 3 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.492 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:330/600: open fd = 3 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:331/600: open fd = 3 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:332/600: open fd = 3 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:333/600: open fd = 3 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.493 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:334/600: open fd = 3 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:335/600: open fd = 3 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:336/600: open fd = 3 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.494 INFO:tasks.workunit.client.0.smithi037.stdout:337/600: open fd = 3 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:338/600: open fd = 3 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:339/600: open fd = 3 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:340/600: open fd = 3 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.495 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:341/600: open fd = 3 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:342/600: open fd = 3 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:343/600: open fd = 3 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:344/600: open fd = 3 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.496 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:345/600: open fd = 3 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:346/600: open fd = 3 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:347/600: open fd = 3 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:348/600: open fd = 3 2024-04-25T21:19:37.497 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:349/600: open fd = 3 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:350/600: open fd = 3 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:351/600: open fd = 3 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.498 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:352/600: open fd = 3 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:353/600: open fd = 3 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:354/600: open fd = 3 2024-04-25T21:19:37.499 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:355/600: open fd = 3 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:356/600: open fd = 3 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:357/600: open fd = 3 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.500 INFO:tasks.workunit.client.0.smithi037.stdout:358/600: open fd = 3 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:359/600: open fd = 3 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:360/600: open fd = 3 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:361/600: open fd = 3 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.501 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:362/600: open fd = 3 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:363/600: open fd = 3 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:364/600: open fd = 3 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:365/600: open fd = 3 2024-04-25T21:19:37.502 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:366/600: open fd = 3 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:367/600: open fd = 3 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:368/600: open fd = 3 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.503 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:369/600: open fd = 3 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:370/600: open fd = 3 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:371/600: open fd = 3 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:372/600: open fd = 3 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.504 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:373/600: open fd = 3 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:374/600: open fd = 3 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:19:37.505 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:19:37.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:37 smithi098 ceph-mon[32569]: pgmap v347: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:37.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:37 smithi037 ceph-mon[29195]: pgmap v347: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:37 smithi177 ceph-mon[33043]: pgmap v347: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:39.919 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:39 smithi098 ceph-mon[32569]: pgmap v348: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:39 smithi037 ceph-mon[29195]: pgmap v348: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:39 smithi177 ceph-mon[33043]: pgmap v348: 97 pgs: 97 active+clean; 2.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 -- 172.21.15.37:0/3021996454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c103b20 msgr2=0x7fc53c10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 --2- 172.21.15.37:0/3021996454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c103b20 0x7fc53c10c000 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc530009900 tx=0x7fc53002a480 comp rx=0 tx=0).stop 2024-04-25T21:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 -- 172.21.15.37:0/3021996454 shutdown_connections 2024-04-25T21:19:41.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 --2- 172.21.15.37:0/3021996454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c10c5d0 0x7fc53c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 --2- 172.21.15.37:0/3021996454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c103b20 0x7fc53c10c000 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 --2- 172.21.15.37:0/3021996454 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103200 0x7fc53c1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 -- 172.21.15.37:0/3021996454 >> 172.21.15.37:0/3021996454 conn(0x7fc53c0771d0 msgr2=0x7fc53c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 -- 172.21.15.37:0/3021996454 shutdown_connections 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.230+0000 7fc541d68640 1 -- 172.21.15.37:0/3021996454 wait complete. 2024-04-25T21:19:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 Processor -- start 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 -- start start 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 0x7fc53c100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 0x7fc53c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1121f0 con 0x7fc53c10c5d0 2024-04-25T21:19:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112070 con 0x7fc53c103b20 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc541d68640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112370 con 0x7fc53c103200 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc53bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc53bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56424/0 (socket says 172.21.15.37:56424) 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc53bfff640 1 -- 172.21.15.37:0/2212127082 learned_addr learned my addr 172.21.15.37:0/2212127082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 0x7fc53c100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.231+0000 7fc53affd640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 0x7fc53c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 -- 172.21.15.37:0/2212127082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 msgr2=0x7fc53c100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 0x7fc53c100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 -- 172.21.15.37:0/2212127082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 msgr2=0x7fc53c101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 0x7fc53c101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 -- 172.21.15.37:0/2212127082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1ab070 con 0x7fc53c10c5d0 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 0x7fc53c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53affd640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 0x7fc53c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc53bfff640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc53400dac0 tx=0x7fc53400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc538ff9640 1 -- 172.21.15.37:0/2212127082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534017070 con 0x7fc53c10c5d0 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1ab2a0 con 0x7fc53c10c5d0 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc538ff9640 1 -- 172.21.15.37:0/2212127082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc53400be30 con 0x7fc53c10c5d0 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc538ff9640 1 -- 172.21.15.37:0/2212127082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534010460 con 0x7fc53c10c5d0 2024-04-25T21:19:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.232+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1ab770 con 0x7fc53c10c5d0 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc538ff9640 1 -- 172.21.15.37:0/2212127082 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5340040b0 con 0x7fc53c10c5d0 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 Processor -- start 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 -- start start 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d55a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d62bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556545cba970 con 0x556545d55f40 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556545c574a0 con 0x556545d55620 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc544a80740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556545cb6930 con 0x556545d60800 2024-04-25T21:19:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc538ff9640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 0x7fc5240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56440/0 (socket says 172.21.15.37:56440) 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc53b7fe640 1 -- 172.21.15.37:0/258630 learned_addr learned my addr 172.21.15.37:0/258630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.235+0000 7fc53bfff640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 0x7fc5240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53affd640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d62bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc538ff9640 1 -- 172.21.15.37:0/2212127082 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc5340d01e0 con 0x7fc53c10c5d0 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53affd640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d55a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 -- 172.21.15.37:0/258630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 msgr2=0x556545d62bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d62bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 -- 172.21.15.37:0/258630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 msgr2=0x556545d55a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d55a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 -- 172.21.15.37:0/258630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565459cd210 con 0x556545d55f40 2024-04-25T21:19:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53affd640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bcc0 tx=0x7fc52c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede4979e3dad73d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc53bfff640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 0x7fc5240a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5340cd9e0 tx=0x7fc53406ec70 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc521ffb640 1 -- 172.21.15.37:0/258630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c005020 con 0x556545d55f40 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc521ffb640 1 -- 172.21.15.37:0/258630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0047a0 con 0x556545d55f40 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.236+0000 7fc521ffb640 1 -- 172.21.15.37:0/258630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c011640 con 0x556545d55f40 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- 172.21.15.37:0/258630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 msgr2=0x556545d602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bcc0 tx=0x7fc52c00f730 comp rx=0 tx=0).stop 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- 172.21.15.37:0/258630 shutdown_connections 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d62bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d602c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 --2- 172.21.15.37:0/258630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d55a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- 172.21.15.37:0/258630 >> 172.21.15.37:0/258630 conn(0x556545d51b30 msgr2=0x556545d51f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- 172.21.15.37:0/258630 shutdown_connections 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- 172.21.15.37:0/258630 wait complete. 2024-04-25T21:19:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.237+0000 7fc544a80740 1 -- start start 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 Processor -- start 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 0x556545d5d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556545d63430 con 0x556545d60800 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556545c550b0 con 0x556545d55620 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556545d635b0 con 0x556545d55f40 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc544a80740 10 client.0 _ll_register_callbacks cb 0x556545c42d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56446/0 (socket says 172.21.15.37:56446) 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 -- 172.21.15.37:0/3015134481 learned_addr learned my addr 172.21.15.37:0/3015134481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 0x556545d5d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 -- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 msgr2=0x556545d5d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 0x556545d5d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 -- 172.21.15.37:0/3015134481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 msgr2=0x556545d600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53affd640 1 -- 172.21.15.37:0/3015134481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556545a99620 con 0x556545d60800 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.238+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 0x556545d5d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc53affd640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc53002a9a0 tx=0x7fc53002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8b20830fdb74fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530035040 con 0x556545d60800 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc53003d680 con 0x556545d60800 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc52c0047a0 con 0x556545d60800 2024-04-25T21:19:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc530050050 con 0x556545d60800 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 10 client.16671 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556545d6f280 con 0x556545d60800 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0047a0 con 0x556545d60800 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc53003a040 con 0x556545d60800 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 10 client.16671 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 20 client.16671 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545d706f0 0x556545d72ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 4 client.16671 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.239+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556545cc8390 con 0x556545d706f0 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.240+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545d706f0 0x556545d72ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.240+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545d706f0 0x556545d72ae0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6647b35fc1856943 server_cookie=593f6d4e41462656 in_seq=0 out_seq=0 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.240+0000 7fc51affd640 10 client.16671 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc51affd640 1 -- 172.21.15.37:0/3015134481 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556545cc8390 con 0x556545d706f0 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc51affd640 10 client.16671 handle_command_reply: tid=0 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 client.16671 shutdown 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 msgr2=0x556545d5d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc53002a9a0 tx=0x7fc53002a480 comp rx=0 tx=0).stop 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 shutdown_connections 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545d706f0 0x556545d72ae0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d60800 0x556545d5d8f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d55f40 0x556545d5d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc51affd640 0 client.16671 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 --2- 172.21.15.37:0/3015134481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 >> 172.21.15.37:0/3015134481 conn(0x556545d51b30 msgr2=0x556545d578b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 shutdown_connections 2024-04-25T21:19:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 1 -- 172.21.15.37:0/3015134481 wait complete. 2024-04-25T21:19:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.241+0000 7fc544a80740 20 client.16671 trim_cache size 0 max 16384 2024-04-25T21:19:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 Processor -- start 2024-04-25T21:19:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 -- start start 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d6c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b4050 con 0x556545d55f40 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b5b90 con 0x556545d55620 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc544a80740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b4720 con 0x556545d60800 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56462/0 (socket says 172.21.15.37:56462) 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 -- 172.21.15.37:0/952897254 learned_addr learned my addr 172.21.15.37:0/952897254 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 -- 172.21.15.37:0/952897254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 msgr2=0x556545d783b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53affd640 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d6c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 -- 172.21.15.37:0/952897254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 msgr2=0x556545d6c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d6c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53b7fe640 1 -- 172.21.15.37:0/952897254 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556545d5b900 con 0x556545d55f40 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.257+0000 7fc53affd640 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d6c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bcc0 tx=0x7fc52c00b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3976c1fa61980027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc5217fa640 1 -- 172.21.15.37:0/952897254 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004240 con 0x556545d55f40 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc5217fa640 1 -- 172.21.15.37:0/952897254 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0043e0 con 0x556545d55f40 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc5217fa640 1 -- 172.21.15.37:0/952897254 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c028290 con 0x556545d55f40 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 -- 172.21.15.37:0/952897254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 msgr2=0x556545d5fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00bcc0 tx=0x7fc52c00b9a0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 -- 172.21.15.37:0/952897254 shutdown_connections 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d783b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d5fbf0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 --2- 172.21.15.37:0/952897254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d6c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 -- 172.21.15.37:0/952897254 >> 172.21.15.37:0/952897254 conn(0x556545d51b30 msgr2=0x556545d589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 -- 172.21.15.37:0/952897254 shutdown_connections 2024-04-25T21:19:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.258+0000 7fc544a80740 1 -- 172.21.15.37:0/952897254 wait complete. 2024-04-25T21:19:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 -- start start 2024-04-25T21:19:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 Processor -- start 2024-04-25T21:19:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d68c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b54c0 con 0x556545d55f40 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b4df0 con 0x556545d55620 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c1b2050 con 0x556545d60800 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc544a80740 10 client.0 _ll_register_callbacks cb 0x7fc530031dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc53b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56476/0 (socket says 172.21.15.37:56476) 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.259+0000 7fc53b7fe640 1 -- 172.21.15.37:0/678119224 learned_addr learned my addr 172.21.15.37:0/678119224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53affd640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d68c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 -- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 msgr2=0x556545d64610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 -- 172.21.15.37:0/678119224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 msgr2=0x556545d68c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d68c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 -- 172.21.15.37:0/678119224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556545d64e10 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53affd640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d68c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc53b7fe640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006010 tx=0x7fc52c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a071c5af190e635a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc51affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c004240 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc52c0035f0 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc52c0043e0 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc52c03e050 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc544a80740 10 client.16677 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556545dd6c70 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.260+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc52c0043e0 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc52c0286d0 con 0x556545d55f40 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 10 client.16677 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 20 client.16677 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545dd71f0 0x556545dd9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 4 client.16677 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:19:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556545d5dc80 con 0x556545dd71f0 2024-04-25T21:19:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc53affd640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545dd71f0 0x556545dd9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc51affd640 10 client.16677 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:41.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.261+0000 7fc53affd640 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545dd71f0 0x556545dd9610 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75829d67ce4f78d8 server_cookie=5c30e26929fed429 in_seq=0 out_seq=0 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.275+0000 7fc51affd640 1 -- 172.21.15.37:0/678119224 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556545d5dc80 con 0x556545dd71f0 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.275+0000 7fc51affd640 10 client.16677 handle_command_reply: tid=0 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 client.16677 shutdown 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 msgr2=0x556545d64000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc52c006010 tx=0x7fc52c0047a0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 shutdown_connections 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556545dd71f0 0x556545dd9610 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556545d60800 0x556545d64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc51affd640 0 client.16677 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556545d55f40 0x556545d64000 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 --2- 172.21.15.37:0/678119224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556545d55620 0x556545d68c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 >> 172.21.15.37:0/678119224 conn(0x556545d51b30 msgr2=0x556545d58100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 shutdown_connections 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 1 -- 172.21.15.37:0/678119224 wait complete. 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.276+0000 7fc544a80740 20 client.16677 trim_cache size 0 max 16384 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:19:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 msgr2=0x7fc5240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 0x7fc5240a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5340cd9e0 tx=0x7fc53406ec70 comp rx=0 tx=0).stop 2024-04-25T21:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 msgr2=0x7fc53c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc53400dac0 tx=0x7fc53400df80 comp rx=0 tx=0).stop 2024-04-25T21:19:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 shutdown_connections 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc53c10c5d0 0x7fc53c1018d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc53c103b20 0x7fc53c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc5240a3dc0 0x7fc5240a6280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 --2- 172.21.15.37:0/2212127082 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc53c103200 0x7fc53c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 >> 172.21.15.37:0/2212127082 conn(0x7fc53c0771d0 msgr2=0x7fc53c0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 shutdown_connections 2024-04-25T21:19:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:41.277+0000 7fc541d68640 1 -- 172.21.15.37:0/2212127082 wait complete. 2024-04-25T21:19:41.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:41 smithi098 ceph-mon[32569]: pgmap v349: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:41 smithi037 ceph-mon[29195]: pgmap v349: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:41 smithi177 ceph-mon[33043]: pgmap v349: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:43 smithi098 ceph-mon[32569]: pgmap v350: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:19:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:43 smithi037 ceph-mon[29195]: pgmap v350: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:19:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:43 smithi177 ceph-mon[33043]: pgmap v350: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:19:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:45 smithi098 ceph-mon[32569]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:19:45.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:45 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:19:45.975 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.1 2024-04-25T21:19:45.976 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:19:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:45 smithi037 ceph-mon[29195]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:19:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:45 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:19:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:45 smithi177 ceph-mon[33043]: pgmap v351: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-25T21:19:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:45 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:19:46.236 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:19:46.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:46 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:19:46.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:46 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:19:46.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:46 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:19:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:46 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:19:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:46 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:19:46.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:46 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:19:47.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:46 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:19:47.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:46 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:19:47.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:46 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:19:47.770 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:47 smithi037 ceph-mon[29195]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:47.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:47 smithi098 ceph-mon[32569]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:47 smithi177 ceph-mon[33043]: pgmap v352: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1499105870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4109f70 msgr2=0x7fc6c4111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:48.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1499105870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4109f70 0x7fc6c4111b40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00b0d0 tx=0x7fc6bc00b3e0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1499105870 shutdown_connections 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1499105870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4109f70 0x7fc6c4111b40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1499105870 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4104e90 0x7fc6c4105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1499105870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 0x7fc6c4109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1499105870 >> 172.21.15.37:0/1499105870 conn(0x7fc6c4100720 msgr2=0x7fc6c4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1499105870 shutdown_connections 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1499105870 wait complete. 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.287+0000 7fc6ca2e3700 1 Processor -- start 2024-04-25T21:19:48.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 -- start start 2024-04-25T21:19:48.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:48.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 0x7fc6c419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:48.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4109f70 0x7fc6c419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114e20 con 0x7fc6c4104e90 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114ca0 con 0x7fc6c4109f70 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6ca2e3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4114fa0 con 0x7fc6c4105840 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6c3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6c3fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56492/0 (socket says 172.21.15.37:56492) 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6c3fff700 1 -- 172.21.15.37:0/1372241255 learned_addr learned my addr 172.21.15.37:0/1372241255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6c8880700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4109f70 0x7fc6c419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.288+0000 7fc6c37fe700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 0x7fc6c419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 -- 172.21.15.37:0/1372241255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 msgr2=0x7fc6c419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 0x7fc6c419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 -- 172.21.15.37:0/1372241255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4109f70 msgr2=0x7fc6c419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4109f70 0x7fc6c419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 -- 172.21.15.37:0/1372241255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c41a02e0 con 0x7fc6c4104e90 2024-04-25T21:19:48.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c3fff700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e9a0 tx=0x7fc6b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400ca10 con 0x7fc6c4104e90 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6b400cbb0 con 0x7fc6c4104e90 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c41a05d0 con 0x7fc6c4104e90 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4018680 con 0x7fc6c4104e90 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.289+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c41a7e50 con 0x7fc6c4104e90 2024-04-25T21:19:48.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.290+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c4069730 con 0x7fc6c4104e90 2024-04-25T21:19:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.291+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc6b4010430 con 0x7fc6c4104e90 2024-04-25T21:19:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.292+0000 7fc6c17fa700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 0x7fc6ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.292+0000 7fc6c37fe700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 0x7fc6ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.292+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fc6b40d0990 con 0x7fc6c4104e90 2024-04-25T21:19:48.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.292+0000 7fc6c37fe700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 0x7fc6ac0a6270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4103e20 tx=0x7fc6b8009260 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:48.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.297+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6b409d030 con 0x7fc6c4104e90 2024-04-25T21:19:48.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.552+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6c402df80 con 0x7fc6c4104e90 2024-04-25T21:19:48.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.553+0000 7fc6c17fa700 1 -- 172.21.15.37:0/1372241255 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fc6b409c930 con 0x7fc6c4104e90 2024-04-25T21:19:48.553 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:19:48.553 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 msgr2=0x7fc6ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 0x7fc6ac0a6270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4103e20 tx=0x7fc6b8009260 comp rx=0 tx=0).stop 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 msgr2=0x7fc6c419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400e9a0 tx=0x7fc6b400ecb0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 shutdown_connections 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc6ac0a3db0 0x7fc6ac0a6270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc6c4109f70 0x7fc6c419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6c4104e90 0x7fc6c419b2d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 --2- 172.21.15.37:0/1372241255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6c4105840 0x7fc6c419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:48.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 >> 172.21.15.37:0/1372241255 conn(0x7fc6c4100720 msgr2=0x7fc6c4102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:48.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 shutdown_connections 2024-04-25T21:19:48.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:48.555+0000 7fc6ca2e3700 1 -- 172.21.15.37:0/1372241255 wait complete. 2024-04-25T21:19:48.556 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:19:48.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:48 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1372241255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:19:48.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:48 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1372241255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:19:49.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:48 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1372241255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:19:49.230 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 88.1 --await-for 42.0' 2024-04-25T21:19:49.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 88.1 --await-for 42.0 2024-04-25T21:19:49.529 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:19:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:49 smithi098 ceph-mon[32569]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:49.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:49 smithi037 ceph-mon[29195]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:49.990 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:49 smithi177 ceph-mon[33043]: pgmap v353: 97 pgs: 97 active+clean; 3.3 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:19:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 -- 172.21.15.37:0/2100328959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 msgr2=0x7f7e3c102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 --2- 172.21.15.37:0/2100328959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c102110 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009990 tx=0x7f7e2c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 -- 172.21.15.37:0/2100328959 shutdown_connections 2024-04-25T21:19:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 --2- 172.21.15.37:0/2100328959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 --2- 172.21.15.37:0/2100328959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 --2- 172.21.15.37:0/2100328959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c102110 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 -- 172.21.15.37:0/2100328959 >> 172.21.15.37:0/2100328959 conn(0x7f7e3c0777b0 msgr2=0x7f7e3c077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 -- 172.21.15.37:0/2100328959 shutdown_connections 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.275+0000 7f7e4417e700 1 -- 172.21.15.37:0/2100328959 wait complete. 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 Processor -- start 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 -- start start 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111b00 con 0x7f7e3c101c90 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111980 con 0x7f7e3c102650 2024-04-25T21:19:51.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4417e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111c80 con 0x7f7e3c106930 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39190/0 (socket says 172.21.15.37:39190) 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 -- 172.21.15.37:0/3036190923 learned_addr learned my addr 172.21.15.37:0/3036190923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41719700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4271b700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 -- 172.21.15.37:0/3036190923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 msgr2=0x7f7e3c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 -- 172.21.15.37:0/3036190923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 msgr2=0x7f7e3c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41f1a700 1 -- 172.21.15.37:0/3036190923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c19dfe0 con 0x7f7e3c101c90 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e41719700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.276+0000 7f7e4271b700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d980 tx=0x7f7e3800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e32ffd700 1 -- 172.21.15.37:0/3036190923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38016070 con 0x7f7e3c101c90 2024-04-25T21:19:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e32ffd700 1 -- 172.21.15.37:0/3036190923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e3800bb60 con 0x7f7e3c101c90 2024-04-25T21:19:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e32ffd700 1 -- 172.21.15.37:0/3036190923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800f3c0 con 0x7f7e3c101c90 2024-04-25T21:19:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c19e2d0 con 0x7f7e3c101c90 2024-04-25T21:19:51.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.277+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a5b50 con 0x7f7e3c101c90 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.279+0000 7f7e32ffd700 1 -- 172.21.15.37:0/3036190923 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e38019070 con 0x7f7e3c101c90 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.280+0000 7f7e32ffd700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 0x7f7e280a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.280+0000 7f7e41719700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 0x7f7e280a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.280+0000 7f7e32ffd700 1 -- 172.21.15.37:0/3036190923 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f7e380d04e0 con 0x7f7e3c101c90 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e41719700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 0x7f7e280a6390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006010 tx=0x7f7e2c00b540 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 Processor -- start 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 -- start start 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa793e1ec0 con 0x55aa7948ea80 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aa79388af0 con 0x55aa7949e680 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.281+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55aa793dde80 con 0x55aa7948f3a0 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39194/0 (socket says 172.21.15.37:39194) 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 -- 172.21.15.37:0/2763066537 learned_addr learned my addr 172.21.15.37:0/2763066537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e4271b700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41719700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 -- 172.21.15.37:0/2763066537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 msgr2=0x55aa7949e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 -- 172.21.15.37:0/2763066537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 msgr2=0x55aa794a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 -- 172.21.15.37:0/2763066537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7932fa60 con 0x55aa7948ea80 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41719700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7e38098df0 tx=0x7f7e38096f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8584e8739b97813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e27fff700 1 -- 172.21.15.37:0/2763066537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a07d0 con 0x55aa7948ea80 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e27fff700 1 -- 172.21.15.37:0/2763066537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e380a5090 con 0x55aa7948ea80 2024-04-25T21:19:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.282+0000 7f7e27fff700 1 -- 172.21.15.37:0/2763066537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3809c8e0 con 0x55aa7948ea80 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/2763066537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 msgr2=0x55aa7948ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7e38098df0 tx=0x7f7e38096f90 comp rx=0 tx=0).stop 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/2763066537 shutdown_connections 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa7948ee60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/2763066537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/2763066537 >> 172.21.15.37:0/2763066537 conn(0x55aa793bc820 msgr2=0x55aa7948af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/2763066537 shutdown_connections 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.283+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/2763066537 wait complete. 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 -- start start 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 Processor -- start 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7948ea80 0x55aa7949dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55aa794a1010 con 0x55aa7949e680 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55aa79380d60 con 0x55aa7948ea80 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55aa794a1190 con 0x55aa7948f3a0 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4ea07b80 10 client.0 _ll_register_callbacks cb 0x55aa7937aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4271b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4271b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39208/0 (socket says 172.21.15.37:39208) 2024-04-25T21:19:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e4271b700 1 -- 172.21.15.37:0/4239408864 learned_addr learned my addr 172.21.15.37:0/4239408864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e41719700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.284+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7948ea80 0x55aa7949dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 -- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 msgr2=0x55aa7949b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 -- 172.21.15.37:0/4239408864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7948ea80 msgr2=0x55aa7949dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7948ea80 0x55aa7949dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 -- 172.21.15.37:0/4239408864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa79399400 con 0x55aa7949e680 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4271b700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7e34008f20 tx=0x7f7e34010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3377a4f71a372a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e257fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34010da0 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f7e34016470 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e380a5090 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e34024050 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4ea07b80 10 client.16695 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.285+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa794ab880 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e380a5090 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e340155e0 con 0x55aa7949e680 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 10 client.16695 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 10 client.16695 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 20 client.16695 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa794ade10 0x55aa794b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 4 client.16695 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa793ef6b0 con 0x55aa794ade10 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.286+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa794ade10 0x55aa794b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.287+0000 7f7e257fa700 10 client.16695 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.287+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa794ade10 0x55aa794b0230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b25c4880c7ec196 server_cookie=ee119b4ad3920b04 in_seq=0 out_seq=0 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.287+0000 7f7e257fa700 1 -- 172.21.15.37:0/4239408864 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa793ef6b0 con 0x55aa794ade10 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.287+0000 7f7e257fa700 10 client.16695 handle_command_reply: tid=0 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.287+0000 7f7e4ea07b80 1 client.16695 shutdown 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 msgr2=0x55aa7949b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7e34008f20 tx=0x7f7e34010430 comp rx=0 tx=0).stop 2024-04-25T21:19:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 shutdown_connections 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa794ade10 0x55aa794b0230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7949e680 0x55aa7949b650 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e257fa700 0 client.16695 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7948ea80 0x55aa7949dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/4239408864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa7949b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 >> 172.21.15.37:0/4239408864 conn(0x55aa793bc820 msgr2=0x55aa7948ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 shutdown_connections 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/4239408864 wait complete. 2024-04-25T21:19:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.288+0000 7f7e4ea07b80 20 client.16695 trim_cache size 0 max 16384 2024-04-25T21:19:51.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.307+0000 7f7e4ea07b80 1 Processor -- start 2024-04-25T21:19:51.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.307+0000 7f7e4ea07b80 1 -- start start 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948ea80 0x55aa79496970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa7949d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1ab080 con 0x55aa7948f3a0 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1acbc0 con 0x55aa7949e680 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1ab750 con 0x55aa7948ea80 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39216/0 (socket says 172.21.15.37:39216) 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 -- 172.21.15.37:0/3442144988 learned_addr learned my addr 172.21.15.37:0/3442144988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:51.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948ea80 0x55aa79496970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4271b700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa7949d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 -- 172.21.15.37:0/3442144988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948ea80 msgr2=0x55aa79496970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948ea80 0x55aa79496970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 -- 172.21.15.37:0/3442144988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 msgr2=0x55aa7949d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa7949d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 -- 172.21.15.37:0/3442144988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa794b0890 con 0x55aa7948f3a0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e4271b700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa7949d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.308+0000 7f7e41719700 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009990 tx=0x7f7e2c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5614cda788e44c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e277fe700 1 -- 172.21.15.37:0/3442144988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c035070 con 0x55aa7948f3a0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e277fe700 1 -- 172.21.15.37:0/3442144988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e2c005030 con 0x55aa7948f3a0 2024-04-25T21:19:51.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e277fe700 1 -- 172.21.15.37:0/3442144988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00f9a0 con 0x55aa7948f3a0 2024-04-25T21:19:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/3442144988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 msgr2=0x55aa794a9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009990 tx=0x7f7e2c022000 comp rx=0 tx=0).stop 2024-04-25T21:19:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/3442144988 shutdown_connections 2024-04-25T21:19:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa7949d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948ea80 0x55aa79496970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/3442144988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948f3a0 0x55aa794a9ed0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/3442144988 >> 172.21.15.37:0/3442144988 conn(0x55aa793bc820 msgr2=0x55aa7949c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/3442144988 shutdown_connections 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.309+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/3442144988 wait complete. 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 -- start start 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 Processor -- start 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa794a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1ac4f0 con 0x55aa7948ea80 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1abe20 con 0x55aa7949e680 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1a9040 con 0x55aa7948f3a0 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e4ea07b80 10 client.0 _ll_register_callbacks cb 0x55aa7937aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e41f1a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39226/0 (socket says 172.21.15.37:39226) 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e41f1a700 1 -- 172.21.15.37:0/929737629 learned_addr learned my addr 172.21.15.37:0/929737629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.310+0000 7f7e41719700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa794a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 -- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 msgr2=0x55aa794a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa794a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e4271b700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 -- 172.21.15.37:0/929737629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 msgr2=0x55aa794a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:51.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 -- 172.21.15.37:0/929737629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa794a1fb0 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e4271b700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e41f1a700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b540 tx=0x7f7e38098790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9d0e910fdcd30d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e257fa700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380ba060 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7f7e38098a90 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e2c005030 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e38085070 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e4ea07b80 10 client.16701 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa79515170 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.311+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e2c005030 con 0x55aa7948ea80 2024-04-25T21:19:51.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7e3809c680 con 0x55aa7948ea80 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 10 client.16701 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 20 client.16701 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa79515460 0x55aa79517850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 4 client.16701 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 88.1, "await_for": 42.0}] 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 88.1, "await_for": 42.0}) -- 0x55aa793e9750 con 0x55aa79515460 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e41719700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa79515460 0x55aa79517850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e41719700 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa79515460 0x55aa79517850 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca11f7639b27d7 server_cookie=4bb5105f09b8acb7 in_seq=0 out_seq=0 2024-04-25T21:19:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.312+0000 7f7e257fa700 10 client.16701 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.327+0000 7f7e257fa700 1 -- 172.21.15.37:0/929737629 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55aa793e9750 con 0x55aa79515460 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.327+0000 7f7e257fa700 10 client.16701 handle_command_reply: tid=0 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.327+0000 7f7e4ea07b80 1 client.16701 shutdown 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.327+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 msgr2=0x55aa794a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.327+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800b540 tx=0x7f7e38098790 comp rx=0 tx=0).stop 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 shutdown_connections 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55aa79515460 0x55aa79517850 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55aa7949e680 0x55aa794a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aa7948ea80 0x55aa794a66b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e257fa700 0 client.16701 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 --2- 172.21.15.37:0/929737629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aa7948f3a0 0x55aa794a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 >> 172.21.15.37:0/929737629 conn(0x55aa793bc820 msgr2=0x55aa7949d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 shutdown_connections 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 1 -- 172.21.15.37:0/929737629 wait complete. 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4ea07b80 20 client.16701 trim_cache size 0 max 16384 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:19:51.329 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 30, 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "8fda637f": { 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 30, 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:19:51.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 88.1, 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:51.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:19:51.332 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:19:51.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 msgr2=0x7f7e280a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 0x7f7e280a6390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006010 tx=0x7f7e2c00b540 comp rx=0 tx=0).stop 2024-04-25T21:19:51.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 msgr2=0x7f7e3c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d980 tx=0x7f7e3800dd40 comp rx=0 tx=0).stop 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 shutdown_connections 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f7e280a3ed0 0x7f7e280a6390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e3c106930 0x7f7e3c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e3c101c90 0x7f7e3c199020 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 --2- 172.21.15.37:0/3036190923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7e3c102650 0x7f7e3c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.328+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 >> 172.21.15.37:0/3036190923 conn(0x7f7e3c0777b0 msgr2=0x7f7e3c06b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.329+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 shutdown_connections 2024-04-25T21:19:51.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:19:51.329+0000 7f7e4417e700 1 -- 172.21.15.37:0/3036190923 wait complete. 2024-04-25T21:19:51.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:51 smithi098 ceph-mon[32569]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:51 smithi037 ceph-mon[29195]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:52.050 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8fda637f', quiesce took 6.1 seconds. Will release after: 28.1 2024-04-25T21:19:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:51 smithi177 ceph-mon[33043]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:19:53.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:53 smithi098 ceph-mon[32569]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-25T21:19:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:53 smithi037 ceph-mon[29195]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-25T21:19:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:53 smithi177 ceph-mon[33043]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3649148968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e01061b0 msgr2=0x7fa9e010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3649148968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e01061b0 0x7fa9e010dd80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009ca0 tx=0x7fa9d802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3649148968 shutdown_connections 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3649148968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e01061b0 0x7fa9e010dd80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3649148968 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e0101a90 0x7fa9e0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3649148968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 0x7fa9e01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3649148968 >> 172.21.15.37:0/3649148968 conn(0x7fa9e00771d0 msgr2=0x7fa9e00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3649148968 shutdown_connections 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3649148968 wait complete. 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 Processor -- start 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.379+0000 7fa9e6e0b640 1 -- start start 2024-04-25T21:19:54.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 0x7fa9e01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e01061b0 0x7fa9e0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0112230 con 0x7fa9e019ce60 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01120b0 con 0x7fa9e01010e0 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e6e0b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01123b0 con 0x7fa9e01061b0 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39242/0 (socket says 172.21.15.37:39242) 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 -- 172.21.15.37:0/3561538113 learned_addr learned my addr 172.21.15.37:0/3561538113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 0x7fa9e01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9dffff640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e01061b0 0x7fa9e0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 -- 172.21.15.37:0/3561538113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e01061b0 msgr2=0x7fa9e0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e01061b0 0x7fa9e0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 -- 172.21.15.37:0/3561538113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 msgr2=0x7fa9e01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 0x7fa9e01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 -- 172.21.15.37:0/3561538113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e019d950 con 0x7fa9e019ce60 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.380+0000 7fa9e53a8640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005cc0 tx=0x7fa9d802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.381+0000 7fa9ddffb640 1 -- 172.21.15.37:0/3561538113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8003bf0 con 0x7fa9e019ce60 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.381+0000 7fa9ddffb640 1 -- 172.21.15.37:0/3561538113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d8003d90 con 0x7fa9e019ce60 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.381+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e019dbe0 con 0x7fa9e019ce60 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.381+0000 7fa9ddffb640 1 -- 172.21.15.37:0/3561538113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d802fbc0 con 0x7fa9e019ce60 2024-04-25T21:19:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.381+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e01a5430 con 0x7fa9e019ce60 2024-04-25T21:19:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.383+0000 7fa9ddffb640 1 -- 172.21.15.37:0/3561538113 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9d802fd60 con 0x7fa9e019ce60 2024-04-25T21:19:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 Processor -- start 2024-04-25T21:19:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 -- start start 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27232b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d272290970 con 0x55d27233ac10 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d27222d4a0 con 0x55d27232b540 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d27228c930 con 0x55d27232be60 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39252/0 (socket says 172.21.15.37:39252) 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 -- 172.21.15.37:0/1520325461 learned_addr learned my addr 172.21.15.37:0/1520325461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9ddffb640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 0x7fa9cc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27232b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e53a8640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 0x7fa9cc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 -- 172.21.15.37:0/1520325461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 msgr2=0x55d27233a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 -- 172.21.15.37:0/1520325461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 msgr2=0x55d27232b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27232b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9dffff640 1 -- 172.21.15.37:0/1520325461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d271fa3210 con 0x55d27233ac10 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.384+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9ddffb640 1 -- 172.21.15.37:0/3561538113 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fa9d80f1e10 con 0x7fa9e019ce60 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800cf60 tx=0x7fa9c800c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbfb8f78b14dc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9d6ffd640 1 -- 172.21.15.37:0/1520325461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8019030 con 0x55d27233ac10 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9e53a8640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 0x7fa9cc0a6300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ba320 tx=0x7fa9d80e8fd0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9d6ffd640 1 -- 172.21.15.37:0/1520325461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c80119d0 con 0x55d27233ac10 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.385+0000 7fa9d6ffd640 1 -- 172.21.15.37:0/1520325461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800f9a0 con 0x55d27233ac10 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1520325461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 msgr2=0x55d27233d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800cf60 tx=0x7fa9c800c610 comp rx=0 tx=0).stop 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1520325461 shutdown_connections 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233d000 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1520325461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27232b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1520325461 >> 172.21.15.37:0/1520325461 conn(0x55d272327b30 msgr2=0x55d272327f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1520325461 shutdown_connections 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1520325461 wait complete. 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 -- start start 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.386+0000 7fa9e9bf5740 1 Processor -- start 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232b540 0x55d27233a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27233ac10 0x55d272337dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d27233d840 con 0x55d27232be60 2024-04-25T21:19:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d27222b0b0 con 0x55d27233ac10 2024-04-25T21:19:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d27233d9c0 con 0x55d27232b540 2024-04-25T21:19:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 10 client.0 _ll_register_callbacks cb 0x55d272218d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39266/0 (socket says 172.21.15.37:39266) 2024-04-25T21:19:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 -- 172.21.15.37:0/1049172217 learned_addr learned my addr 172.21.15.37:0/1049172217 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232b540 0x55d27233a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27233ac10 0x55d272337dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 -- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232b540 msgr2=0x55d27233a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232b540 0x55d27233a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 -- 172.21.15.37:0/1049172217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27233ac10 msgr2=0x55d272337dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27233ac10 0x55d272337dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 -- 172.21.15.37:0/1049172217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27206f620 con 0x55d27232be60 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0010840 tx=0x7fa9d0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e729a89f235aac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9bffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000ed40 con 0x55d27232be60 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fa9d001c480 con 0x55d27232be60 2024-04-25T21:19:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.387+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c80119d0 con 0x55d27232be60 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9d001c680 con 0x55d27232be60 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 10 client.16713 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d272349810 con 0x55d27232be60 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c80119d0 con 0x55d27232be60 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa9d00257b0 con 0x55d27232be60 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 10 client.16713 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 20 client.16713 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d27234acd0 0x55d27234d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 4 client.16713 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27229e390 con 0x55d27234acd0 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.388+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d27234acd0 0x55d27234d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.389+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d27234acd0 0x55d27234d0c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cf54d9be5e001 server_cookie=62077d982b70c2bc in_seq=0 out_seq=0 2024-04-25T21:19:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.389+0000 7fa9bffff640 10 client.16713 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9bffff640 1 -- 172.21.15.37:0/1049172217 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d27229e390 con 0x55d27234acd0 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9bffff640 10 client.16713 handle_command_reply: tid=0 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 client.16713 shutdown 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 msgr2=0x55d272337860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0010840 tx=0x7fa9d0010d00 comp rx=0 tx=0).stop 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 shutdown_connections 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d27234acd0 0x55d27234d0c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27233ac10 0x55d272337dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9bffff640 0 client.16713 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232be60 0x55d272337860 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1049172217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232b540 0x55d27233a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 >> 172.21.15.37:0/1049172217 conn(0x55d272327b30 msgr2=0x55d27232d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 shutdown_connections 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1049172217 wait complete. 2024-04-25T21:19:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.390+0000 7fa9e9bf5740 20 client.16713 trim_cache size 0 max 16384 2024-04-25T21:19:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 Processor -- start 2024-04-25T21:19:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 -- start start 2024-04-25T21:19:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232be60 0x55d272326b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27233ac10 0x55d2723529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ab7d0 con 0x55d27232b540 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ad310 con 0x55d27232be60 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01abea0 con 0x55d27233ac10 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39276/0 (socket says 172.21.15.37:39276) 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.405+0000 7fa9dffff640 1 -- 172.21.15.37:0/2495267768 learned_addr learned my addr 172.21.15.37:0/2495267768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232be60 0x55d272326b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 -- 172.21.15.37:0/2495267768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27233ac10 msgr2=0x55d2723529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27233ac10 0x55d2723529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 -- 172.21.15.37:0/2495267768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232be60 msgr2=0x55d272326b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232be60 0x55d272326b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 -- 172.21.15.37:0/2495267768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27232bc90 con 0x55d27232b540 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9dffff640 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800cf60 tx=0x7fa9c800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72371a5b366becb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9d67fc640 1 -- 172.21.15.37:0/2495267768 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8019040 con 0x55d27232b540 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9d67fc640 1 -- 172.21.15.37:0/2495267768 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c801a040 con 0x55d27232b540 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.406+0000 7fa9d67fc640 1 -- 172.21.15.37:0/2495267768 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800fea0 con 0x55d27232b540 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/2495267768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 msgr2=0x55d27233a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800cf60 tx=0x7fa9c800f580 comp rx=0 tx=0).stop 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/2495267768 shutdown_connections 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27233ac10 0x55d2723529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232be60 0x55d272326b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/2495267768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27232b540 0x55d27233a0a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/2495267768 >> 172.21.15.37:0/2495267768 conn(0x55d272327b30 msgr2=0x55d272326420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/2495267768 shutdown_connections 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/2495267768 wait complete. 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- start start 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 Processor -- start 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27233e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01acc40 con 0x55d27233ac10 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01ac570 con 0x55d27232b540 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.407+0000 7fa9e9bf5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e01a97d0 con 0x55d27232be60 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9e9bf5740 10 client.0 _ll_register_callbacks cb 0x7fa9d00103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39288/0 (socket says 172.21.15.37:39288) 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 -- 172.21.15.37:0/1150418714 learned_addr learned my addr 172.21.15.37:0/1150418714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:19:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27233e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 -- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 msgr2=0x55d27233ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 -- 172.21.15.37:0/1150418714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 msgr2=0x55d27233e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27233e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 -- 172.21.15.37:0/1150418714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2723b02f0 con 0x55d27233ac10 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9dffff640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005fe0 tx=0x7fa9c8013510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f99957479e19c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9bffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8033040 con 0x55d27233ac10 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8513+0+0 (secure 0 0 0) 0x7fa9c8003890 con 0x55d27233ac10 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c801a040 con 0x55d27233ac10 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9c8003a90 con 0x55d27233ac10 2024-04-25T21:19:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9e9bf5740 10 client.16719 fetch_fsmap learned FSMap version 27 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.408+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2723b15e0 con 0x55d27233ac10 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c801a040 con 0x55d27233ac10 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa9c8011740 con 0x55d27233ac10 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 10 client.16719 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 20 client.16719 populate_metadata read hostname 'smithi037' 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2723b1a00 0x55d2723b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 4 client.16719 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d272298200 con 0x55d2723b1a00 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.409+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2723b1a00 0x55d2723b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.410+0000 7fa9e4ba7640 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2723b1a00 0x55d2723b3df0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5166b172147dfb0c server_cookie=e4410f39c59c8f63 in_seq=0 out_seq=0 2024-04-25T21:19:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.410+0000 7fa9bffff640 10 client.16719 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9bffff640 1 -- 172.21.15.37:0/1150418714 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d272298200 con 0x55d2723b1a00 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9bffff640 10 client.16719 handle_command_reply: tid=0 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 client.16719 shutdown 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 msgr2=0x55d27233f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005fe0 tx=0x7fa9c8013510 comp rx=0 tx=0).stop 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 shutdown_connections 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d2723b1a00 0x55d2723b3df0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d27233ac10 0x55d27233f160 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d27232be60 0x55d27233ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 --2- 172.21.15.37:0/1150418714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d27232b540 0x55d27233e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9bffff640 0 client.16719 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 >> 172.21.15.37:0/1150418714 conn(0x55d272327b30 msgr2=0x55d272326cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 shutdown_connections 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 1 -- 172.21.15.37:0/1150418714 wait complete. 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.425+0000 7fa9e9bf5740 20 client.16719 trim_cache size 0 max 16384 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:19:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:19:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 msgr2=0x7fa9cc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 0x7fa9cc0a6300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80ba320 tx=0x7fa9d80e8fd0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 msgr2=0x7fa9e019d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:19:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005cc0 tx=0x7fa9d802f380 comp rx=0 tx=0).stop 2024-04-25T21:19:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 shutdown_connections 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e019ce60 0x7fa9e019d310 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa9cc0a3e40 0x7fa9cc0a6300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9e01061b0 0x7fa9e0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 --2- 172.21.15.37:0/3561538113 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9e01010e0 0x7fa9e01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 >> 172.21.15.37:0/3561538113 conn(0x7fa9e00771d0 msgr2=0x7fa9e00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 shutdown_connections 2024-04-25T21:19:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:19:54.426+0000 7fa9e6e0b640 1 -- 172.21.15.37:0/3561538113 wait complete. 2024-04-25T21:19:55.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:55 smithi098 ceph-mon[32569]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-25T21:19:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:55 smithi037 ceph-mon[29195]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-25T21:19:56.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:55 smithi177 ceph-mon[33043]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-25T21:19:57.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:57 smithi098 ceph-mon[32569]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:19:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:57 smithi037 ceph-mon[29195]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:19:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:57 smithi177 ceph-mon[33043]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:19:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:19:59 smithi037 ceph-mon[29195]: pgmap v358: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:20:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:19:59 smithi177 ceph-mon[33043]: pgmap v358: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:20:00.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:19:59 smithi098 ceph-mon[32569]: pgmap v358: 97 pgs: 97 active+clean; 3.6 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-04-25T21:20:00.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:00 smithi037 ceph-mon[29195]: overall HEALTH_OK 2024-04-25T21:20:01.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:00 smithi177 ceph-mon[33043]: overall HEALTH_OK 2024-04-25T21:20:01.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:00 smithi098 ceph-mon[32569]: overall HEALTH_OK 2024-04-25T21:20:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:01 smithi037 ceph-mon[29195]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T21:20:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:01 smithi177 ceph-mon[33043]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T21:20:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:01 smithi098 ceph-mon[32569]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-25T21:20:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:03 smithi037 ceph-mon[29195]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-25T21:20:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:03 smithi177 ceph-mon[33043]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-25T21:20:04.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:03 smithi098 ceph-mon[32569]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-25T21:20:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:05 smithi037 ceph-mon[29195]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 34 op/s 2024-04-25T21:20:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:05 smithi177 ceph-mon[33043]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 34 op/s 2024-04-25T21:20:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:05 smithi098 ceph-mon[32569]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 64 KiB/s wr, 34 op/s 2024-04-25T21:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2394471193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 msgr2=0x7f3d04104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2394471193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d04104dc0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009ca0 tx=0x7f3cfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2394471193 shutdown_connections 2024-04-25T21:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2394471193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d04104dc0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2394471193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d0406b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2394471193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 0x7f3d0406a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2394471193 >> 172.21.15.37:0/2394471193 conn(0x7f3d04076c50 msgr2=0x7f3d04077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2394471193 shutdown_connections 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.526+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2394471193 wait complete. 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 Processor -- start 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 -- start start 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d041a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 0x7f3d0419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041187a0 con 0x7f3d04104940 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118620 con 0x7f3d0406acc0 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0c09a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04118920 con 0x7f3d04106fa0 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38436/0 (socket says 172.21.15.37:38436) 2024-04-25T21:20:07.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 -- 172.21.15.37:0/2113667947 learned_addr learned my addr 172.21.15.37:0/2113667947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09e36640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d041a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d0a637640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 0x7f3d0419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 -- 172.21.15.37:0/2113667947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 msgr2=0x7f3d0419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 0x7f3d0419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 -- 172.21.15.37:0/2113667947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 msgr2=0x7f3d041a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d041a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09635640 1 -- 172.21.15.37:0/2113667947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0419f060 con 0x7f3d04104940 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.527+0000 7f3d09e36640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d041a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3d09635640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400d950 tx=0x7f3cf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3cfaffd640 1 -- 172.21.15.37:0/2113667947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4017070 con 0x7f3d04104940 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3cfaffd640 1 -- 172.21.15.37:0/2113667947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cf4009c30 con 0x7f3d04104940 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0419f350 con 0x7f3d04104940 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3cfaffd640 1 -- 172.21.15.37:0/2113667947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400f620 con 0x7f3d04104940 2024-04-25T21:20:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.528+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d041ab5a0 con 0x7f3d04104940 2024-04-25T21:20:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.530+0000 7f3cfaffd640 1 -- 172.21.15.37:0/2113667947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3cf4010460 con 0x7f3d04104940 2024-04-25T21:20:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.530+0000 7f3d0edb2740 1 Processor -- start 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 -- start start 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b4522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 0x56373b454bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56373b3ac970 con 0x56373b447620 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56373b3494a0 con 0x56373b447f40 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56373b3a8930 con 0x56373b452800 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3cfaffd640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 0x7f3cf00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38450/0 (socket says 172.21.15.37:38450) 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0a637640 1 -- 172.21.15.37:0/118139132 learned_addr learned my addr 172.21.15.37:0/118139132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d0a637640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 0x7f3cf00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d09e36640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b4522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3cfaffd640 1 -- 172.21.15.37:0/2113667947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f3cf40d0400 con 0x7f3d04104940 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.531+0000 7f3d09635640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 0x56373b454bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 -- 172.21.15.37:0/118139132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 msgr2=0x56373b454bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 0x56373b454bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 -- 172.21.15.37:0/118139132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 msgr2=0x56373b4522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b4522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 -- 172.21.15.37:0/118139132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373b0bf210 con 0x56373b447620 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d09635640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 0x56373b454bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d09e36640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b4522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009a60 tx=0x7f3cfc031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b395900d71feced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3d0a637640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 0x7f3cf00a6300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02f540 tx=0x7f3cfc036000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3ceffff640 1 -- 172.21.15.37:0/118139132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004040 con 0x56373b447620 2024-04-25T21:20:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3ceffff640 1 -- 172.21.15.37:0/118139132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc036070 con 0x56373b447620 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.532+0000 7f3ceffff640 1 -- 172.21.15.37:0/118139132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc02f990 con 0x56373b447620 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- 172.21.15.37:0/118139132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 msgr2=0x56373b447a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009a60 tx=0x7f3cfc031730 comp rx=0 tx=0).stop 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- 172.21.15.37:0/118139132 shutdown_connections 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b452800 0x56373b454bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b4522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/118139132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b447a00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- 172.21.15.37:0/118139132 >> 172.21.15.37:0/118139132 conn(0x56373b443b30 msgr2=0x56373b443f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- 172.21.15.37:0/118139132 shutdown_connections 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- 172.21.15.37:0/118139132 wait complete. 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 -- start start 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.533+0000 7f3d0edb2740 1 Processor -- start 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b4521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b44f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56373b455250 con 0x56373b452800 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56373b3470b0 con 0x56373b447f40 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56373b4553d0 con 0x56373b447620 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0edb2740 10 client.0 _ll_register_callbacks cb 0x56373b334d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38462/0 (socket says 172.21.15.37:38462) 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 -- 172.21.15.37:0/1976494831 learned_addr learned my addr 172.21.15.37:0/1976494831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d09e36640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b4521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d09635640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b44f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 -- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 msgr2=0x56373b4521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b4521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 -- 172.21.15.37:0/1976494831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 msgr2=0x56373b44f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b44f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d0a637640 1 -- 172.21.15.37:0/1976494831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373b18b620 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.534+0000 7f3d09635640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b44f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0a637640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02a900 tx=0x7f3cfc02fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815281439a013835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc060060 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f3cfc004240 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc036070 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3cfc004b40 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 10 client.16731 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56373b455250 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc036070 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3cfc0305c0 con 0x56373b452800 2024-04-25T21:20:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 10 client.16731 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 10 client.16731 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 20 client.16731 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4623d0 0x56373b4647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 4 client.16731 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.535+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373b3ba390 con 0x56373b4623d0 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.536+0000 7f3d09e36640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4623d0 0x56373b4647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.536+0000 7f3ced7fa640 10 client.16731 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.536+0000 7f3d09e36640 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4623d0 0x56373b4647c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe4fe3626476f8f8 server_cookie=e3e5950edba73d97 in_seq=0 out_seq=0 2024-04-25T21:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.537+0000 7f3ced7fa640 1 -- 172.21.15.37:0/1976494831 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56373b3ba390 con 0x56373b4623d0 2024-04-25T21:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.537+0000 7f3ced7fa640 10 client.16731 handle_command_reply: tid=0 2024-04-25T21:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.537+0000 7f3d0edb2740 1 client.16731 shutdown 2024-04-25T21:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.537+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 msgr2=0x56373b44fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.537+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02a900 tx=0x7f3cfc02fda0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 shutdown_connections 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4623d0 0x56373b4647c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b44fac0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b44f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3ced7fa640 0 client.16731 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/1976494831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b4521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 >> 172.21.15.37:0/1976494831 conn(0x56373b443b30 msgr2=0x56373b449860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 shutdown_connections 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 1 -- 172.21.15.37:0/1976494831 wait complete. 2024-04-25T21:20:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.538+0000 7f3d0edb2740 20 client.16731 trim_cache size 0 max 16384 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 Processor -- start 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 -- start start 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b45e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b442b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b3fc0 con 0x56373b452800 2024-04-25T21:20:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b5b00 con 0x56373b447f40 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b4690 con 0x56373b447620 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.553+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38478/0 (socket says 172.21.15.37:38478) 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 -- 172.21.15.37:0/82720057 learned_addr learned my addr 172.21.15.37:0/82720057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d09635640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b442b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d09e36640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b45e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 -- 172.21.15.37:0/82720057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 msgr2=0x56373b45e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b45e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 -- 172.21.15.37:0/82720057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 msgr2=0x56373b442b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b442b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 -- 172.21.15.37:0/82720057 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373b450420 con 0x56373b452800 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d09635640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b442b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3d0a637640 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc030990 tx=0x7f3cfc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32866ea6e57c2866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3cef7fe640 1 -- 172.21.15.37:0/82720057 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc060040 con 0x56373b452800 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3cef7fe640 1 -- 172.21.15.37:0/82720057 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc009540 con 0x56373b452800 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.554+0000 7f3cef7fe640 1 -- 172.21.15.37:0/82720057 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0044f0 con 0x56373b452800 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- 172.21.15.37:0/82720057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 msgr2=0x56373b451cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc030990 tx=0x7f3cfc009a20 comp rx=0 tx=0).stop 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- 172.21.15.37:0/82720057 shutdown_connections 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b452800 0x56373b451cf0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b447f40 0x56373b442b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/82720057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447620 0x56373b45e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- 172.21.15.37:0/82720057 >> 172.21.15.37:0/82720057 conn(0x56373b443b30 msgr2=0x56373b442420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- 172.21.15.37:0/82720057 shutdown_connections 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- 172.21.15.37:0/82720057 wait complete. 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 -- start start 2024-04-25T21:20:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.555+0000 7f3d0edb2740 1 Processor -- start 2024-04-25T21:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 0x56373b44c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 0x56373b44ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b5430 con 0x56373b447620 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b4d60 con 0x56373b452800 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3d041b1fc0 con 0x56373b447f40 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 10 client.0 _ll_register_callbacks cb 0x7f3cfc02fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38480/0 (socket says 172.21.15.37:38480) 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 -- 172.21.15.37:0/107081585 learned_addr learned my addr 172.21.15.37:0/107081585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0a637640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 0x56373b44ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09635640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 0x56373b44c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 -- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 msgr2=0x56373b44c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 0x56373b44c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 -- 172.21.15.37:0/107081585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 msgr2=0x56373b44ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 0x56373b44ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 -- 172.21.15.37:0/107081585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373b4c7900 con 0x56373b447620 2024-04-25T21:20:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0a637640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 0x56373b44ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09635640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 0x56373b44c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d09e36640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004700 tx=0x7f3d00011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef04bf3886d224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3ced7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00011e60 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8557+0+0 (secure 0 0 0) 0x7f3d00004460 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.556+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cfc009540 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d0002c050 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 10 client.16737 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56373b4c8b30 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc009540 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3d00004830 con 0x56373b447620 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 10 client.16737 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 20 client.16737 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4c9280 0x56373b4cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 4 client.16737 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56373b3b4200 con 0x56373b4c9280 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.557+0000 7f3d09635640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4c9280 0x56373b4cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.558+0000 7f3d09635640 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4c9280 0x56373b4cb6a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b3473d5748d34e server_cookie=2b4403bd4f937b4f in_seq=0 out_seq=0 2024-04-25T21:20:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.558+0000 7f3ced7fa640 10 client.16737 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3ced7fa640 1 -- 172.21.15.37:0/107081585 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56373b3b4200 con 0x56373b4c9280 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3ced7fa640 10 client.16737 handle_command_reply: tid=0 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 client.16737 shutdown 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 msgr2=0x56373b44bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004700 tx=0x7f3d00011b50 comp rx=0 tx=0).stop 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 shutdown_connections 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56373b4c9280 0x56373b4cb6a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56373b452800 0x56373b44ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56373b447f40 0x56373b44c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 --2- 172.21.15.37:0/107081585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56373b447620 0x56373b44bfe0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3ced7fa640 0 client.16737 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 >> 172.21.15.37:0/107081585 conn(0x56373b443b30 msgr2=0x56373b442cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 shutdown_connections 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 1 -- 172.21.15.37:0/107081585 wait complete. 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.571+0000 7f3d0edb2740 20 client.16737 trim_cache size 0 max 16384 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:20:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:20:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:20:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 msgr2=0x7f3cf00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 0x7f3cf00a6300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc02f540 tx=0x7f3cfc036000 comp rx=0 tx=0).stop 2024-04-25T21:20:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 msgr2=0x7f3d041a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400d950 tx=0x7f3cf400de10 comp rx=0 tx=0).stop 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 shutdown_connections 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3cf00a3e40 0x7f3cf00a6300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d04106fa0 0x7f3d0419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d04104940 0x7f3d041a4c20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 --2- 172.21.15.37:0/2113667947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d0406acc0 0x7f3d041a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 >> 172.21.15.37:0/2113667947 conn(0x7f3d04076c50 msgr2=0x7f3d04103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 shutdown_connections 2024-04-25T21:20:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:07.572+0000 7f3d0c09a640 1 -- 172.21.15.37:0/2113667947 wait complete. 2024-04-25T21:20:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:07 smithi037 ceph-mon[29195]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-25T21:20:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:07 smithi037 ceph-mon[29195]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:20:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:07 smithi177 ceph-mon[33043]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-25T21:20:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:07 smithi177 ceph-mon[33043]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:20:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:07 smithi098 ceph-mon[32569]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 54 KiB/s wr, 28 op/s 2024-04-25T21:20:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:07 smithi098 ceph-mon[32569]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:20:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:09 smithi037 ceph-mon[29195]: pgmap v364: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-25T21:20:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:09 smithi177 ceph-mon[33043]: pgmap v364: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-25T21:20:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:09 smithi098 ceph-mon[32569]: pgmap v364: 97 pgs: 97 active+clean; 3.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-25T21:20:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:11 smithi037 ceph-mon[29195]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:11 smithi177 ceph-mon[33043]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:11 smithi098 ceph-mon[32569]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:13 smithi037 ceph-mon[29195]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:13 smithi177 ceph-mon[33043]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:13 smithi098 ceph-mon[32569]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 63 KiB/s wr, 33 op/s 2024-04-25T21:20:15.970 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:15 smithi098 ceph-mon[32569]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:15.970 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:15 smithi098 ceph-mon[32569]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:20:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:15 smithi037 ceph-mon[29195]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:15 smithi037 ceph-mon[29195]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:20:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:15 smithi177 ceph-mon[33043]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:15 smithi177 ceph-mon[33043]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:20:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:17 smithi037 ceph-mon[29195]: pgmap v369: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-04-25T21:20:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:17 smithi177 ceph-mon[33043]: pgmap v369: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-04-25T21:20:18.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:17 smithi098 ceph-mon[32569]: pgmap v369: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-04-25T21:20:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:19 smithi037 ceph-mon[29195]: pgmap v370: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:19 smithi177 ceph-mon[33043]: pgmap v370: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:20.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:20:20.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:19 smithi098 ceph-mon[32569]: pgmap v370: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-25T21:20:20.415 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:20:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 -- 172.21.15.37:0/3338359374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d8102b50 msgr2=0x7f81d8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 --2- 172.21.15.37:0/3338359374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d8102b50 0x7f81d8106b90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009900 tx=0x7f81cc02a410 comp rx=0 tx=0).stop 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 -- 172.21.15.37:0/3338359374 shutdown_connections 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 --2- 172.21.15.37:0/3338359374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 0x7f81d810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 --2- 172.21.15.37:0/3338359374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d8102b50 0x7f81d8106b90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 --2- 172.21.15.37:0/3338359374 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d81021a0 0x7f81d8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 -- 172.21.15.37:0/3338359374 >> 172.21.15.37:0/3338359374 conn(0x7f81d80793e0 msgr2=0x7f81d80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 -- 172.21.15.37:0/3338359374 shutdown_connections 2024-04-25T21:20:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.662+0000 7f81dd915640 1 -- 172.21.15.37:0/3338359374 wait complete. 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.663+0000 7f81dd915640 1 Processor -- start 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 -- start start 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 0x7f81d8198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d819cf20 0x7f81d819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8112200 con 0x7f81d81021a0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8112080 con 0x7f81d819cf20 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81dd915640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81d8112380 con 0x7f81d8107270 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57890/0 (socket says 172.21.15.37:57890) 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81d6ffd640 1 -- 172.21.15.37:0/3898981885 learned_addr learned my addr 172.21.15.37:0/3898981885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.664+0000 7f81d77fe640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d819cf20 0x7f81d819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 0x7f81d8198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 -- 172.21.15.37:0/3898981885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 msgr2=0x7f81d8198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 0x7f81d8198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 -- 172.21.15.37:0/3898981885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d819cf20 msgr2=0x7f81d819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d819cf20 0x7f81d819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 -- 172.21.15.37:0/3898981885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d819da10 con 0x7f81d81021a0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f81c800d950 tx=0x7f81c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81dc913640 1 -- 172.21.15.37:0/3898981885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8017030 con 0x7f81d81021a0 2024-04-25T21:20:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81dc913640 1 -- 172.21.15.37:0/3898981885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c8004020 con 0x7f81d81021a0 2024-04-25T21:20:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d819dd00 con 0x7f81d81021a0 2024-04-25T21:20:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81dc913640 1 -- 172.21.15.37:0/3898981885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c80047a0 con 0x7f81d81021a0 2024-04-25T21:20:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.665+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d81a5510 con 0x7f81d81021a0 2024-04-25T21:20:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.668+0000 7f81dc913640 1 -- 172.21.15.37:0/3898981885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f81c8009a30 con 0x7f81d81021a0 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.668+0000 7f81e061f740 1 Processor -- start 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.668+0000 7f81e061f740 1 -- start start 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81dc913640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 0x7f81c00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5618c741e970 con 0x5618c74b9f40 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618c73bb4a0 con 0x5618c74c4800 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81e061f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5618c741a930 con 0x5618c74b9620 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d77fe640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 0x7f81c00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57894/0 (socket says 172.21.15.37:57894) 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:58064/0 (socket says 172.21.15.37:58064) 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 -- 172.21.15.37:0/1436922622 learned_addr learned my addr 172.21.15.37:0/1436922622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81dc913640 1 -- 172.21.15.37:0/3898981885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f81c80d0140 con 0x7f81d81021a0 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 -- 172.21.15.37:0/1436922622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 msgr2=0x5618c74b9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74b9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 -- 172.21.15.37:0/1436922622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 msgr2=0x5618c74c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d67fc640 1 -- 172.21.15.37:0/1436922622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c7131210 con 0x5618c74b9f40 2024-04-25T21:20:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.669+0000 7f81d77fe640 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.670+0000 7f81d77fe640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 0x7f81c00a6300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005fd0 tx=0x7f81d0009d70 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.670+0000 7f81d67fc640 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006010 tx=0x7f81cc004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc3c331077b4f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.670+0000 7f81bdffb640 1 -- 172.21.15.37:0/1436922622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc035040 con 0x5618c74b9f40 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.670+0000 7f81bdffb640 1 -- 172.21.15.37:0/1436922622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81cc0054f0 con 0x5618c74b9f40 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.670+0000 7f81bdffb640 1 -- 172.21.15.37:0/1436922622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0035f0 con 0x5618c74b9f40 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- 172.21.15.37:0/1436922622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 msgr2=0x5618c74c42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006010 tx=0x7f81cc004170 comp rx=0 tx=0).stop 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- 172.21.15.37:0/1436922622 shutdown_connections 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c42c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 --2- 172.21.15.37:0/1436922622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74b9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- 172.21.15.37:0/1436922622 >> 172.21.15.37:0/1436922622 conn(0x5618c74b5b30 msgr2=0x5618c74b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- 172.21.15.37:0/1436922622 shutdown_connections 2024-04-25T21:20:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- 172.21.15.37:0/1436922622 wait complete. 2024-04-25T21:20:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.671+0000 7f81e061f740 1 -- start start 2024-04-25T21:20:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 Processor -- start 2024-04-25T21:20:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5618c74c73d0 con 0x5618c74b9f40 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5618c73b90b0 con 0x5618c74c4800 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5618c74c7550 con 0x5618c74b9620 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81e061f740 10 client.0 _ll_register_callbacks cb 0x5618c73a6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57906/0 (socket says 172.21.15.37:57906) 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 -- 172.21.15.37:0/3122405580 learned_addr learned my addr 172.21.15.37:0/3122405580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d77fe640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 -- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 msgr2=0x5618c74c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 -- 172.21.15.37:0/3122405580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 msgr2=0x5618c74c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 -- 172.21.15.37:0/3122405580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c71fd620 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f81cc005ab0 tx=0x7f81cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf69f9d1cfa8c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.672+0000 7f81b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc053060 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f81cc004ec0 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81cc0054f0 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81cc035050 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 10 client.16749 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618c74d3340 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81cc0054f0 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f81cc003ba0 con 0x5618c74b9f40 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 10 client.16749 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 20 client.16749 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c74d4690 0x5618c74d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 4 client.16749 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.673+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618c742c390 con 0x5618c74d4690 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.674+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c74d4690 0x5618c74d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.674+0000 7f81b6ffd640 10 client.16749 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.674+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c74d4690 0x5618c74d6a80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e9b24301c92ad7 server_cookie=ecadec7f6a4f6044 in_seq=0 out_seq=0 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3122405580 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618c742c390 con 0x5618c74d4690 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81b6ffd640 10 client.16749 handle_command_reply: tid=0 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 client.16749 shutdown 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 msgr2=0x5618c74c1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f81cc005ab0 tx=0x7f81cc005850 comp rx=0 tx=0).stop 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 shutdown_connections 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c74d4690 0x5618c74d6a80 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74c19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81b6ffd640 0 client.16749 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74c1460 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 --2- 172.21.15.37:0/3122405580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9620 0x5618c74c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 >> 172.21.15.37:0/3122405580 conn(0x5618c74b5b30 msgr2=0x5618c74bb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 shutdown_connections 2024-04-25T21:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 1 -- 172.21.15.37:0/3122405580 wait complete. 2024-04-25T21:20:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.675+0000 7f81e061f740 20 client.16749 trim_cache size 0 max 16384 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 Processor -- start 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 -- start start 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 0x5618c74cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81adaa0 con 0x5618c74b9620 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81af5e0 con 0x5618c74c4800 2024-04-25T21:20:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.690+0000 7f81e061f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81ae170 con 0x5618c74b9f40 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57922/0 (socket says 172.21.15.37:57922) 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 -- 172.21.15.37:0/1026928005 learned_addr learned my addr 172.21.15.37:0/1026928005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d67fc640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 0x5618c74cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d77fe640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 -- 172.21.15.37:0/1026928005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 msgr2=0x5618c74cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 0x5618c74cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 -- 172.21.15.37:0/1026928005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 msgr2=0x5618c74ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 -- 172.21.15.37:0/1026928005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618c74c74f0 con 0x5618c74b9620 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d67fc640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 0x5618c74cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f81c8098ab0 tx=0x7f81c809bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8741556e079c5214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81bd7fa640 1 -- 172.21.15.37:0/1026928005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c809c0e0 con 0x5618c74b9620 2024-04-25T21:20:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81bd7fa640 1 -- 172.21.15.37:0/1026928005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c80a0490 con 0x5618c74b9620 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.691+0000 7f81bd7fa640 1 -- 172.21.15.37:0/1026928005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c80cdb10 con 0x5618c74b9620 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 -- 172.21.15.37:0/1026928005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 msgr2=0x5618c74c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f81c8098ab0 tx=0x7f81c809bfe0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 -- 172.21.15.37:0/1026928005 shutdown_connections 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74c4800 0x5618c74ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74b9f40 0x5618c74cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 --2- 172.21.15.37:0/1026928005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9620 0x5618c74c3ca0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 -- 172.21.15.37:0/1026928005 >> 172.21.15.37:0/1026928005 conn(0x5618c74b5b30 msgr2=0x5618c74cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 -- 172.21.15.37:0/1026928005 shutdown_connections 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.692+0000 7f81e061f740 1 -- 172.21.15.37:0/1026928005 wait complete. 2024-04-25T21:20:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 -- start start 2024-04-25T21:20:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 Processor -- start 2024-04-25T21:20:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74b9620 0x5618c74ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74c4800 0x5618c753dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81aef10 con 0x5618c74b9f40 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81ae840 con 0x5618c74b9620 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f81d81abaa0 con 0x5618c74c4800 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81e061f740 10 client.0 _ll_register_callbacks cb 0x7f81cc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57932/0 (socket says 172.21.15.37:57932) 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 -- 172.21.15.37:0/3244103011 learned_addr learned my addr 172.21.15.37:0/3244103011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74b9620 0x5618c74ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d77fe640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74c4800 0x5618c753dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 -- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74c4800 msgr2=0x5618c753dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74c4800 0x5618c753dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 -- 172.21.15.37:0/3244103011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74b9620 msgr2=0x5618c74ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.693+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74b9620 0x5618c74ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81d67fc640 1 -- 172.21.15.37:0/3244103011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618c753e320 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81d67fc640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009900 tx=0x7f81cc02ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfd5c52117db76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc053040 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f81cc0037f0 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c80a0490 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81cc067050 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 10 client.16755 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618c753f630 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c80a0490 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f81cc05e070 con 0x5618c74b9f40 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 10 client.16755 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 20 client.16755 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c753fc90 0x5618c75420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 4 client.16755 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.694+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618c7426200 con 0x5618c753fc90 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.695+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c753fc90 0x5618c75420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.695+0000 7f81d6ffd640 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c753fc90 0x5618c75420b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f2efc47eed92b0 server_cookie=a03fea60076b9151 in_seq=0 out_seq=0 2024-04-25T21:20:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.695+0000 7f81b6ffd640 10 client.16755 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.706+0000 7f81b6ffd640 1 -- 172.21.15.37:0/3244103011 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618c7426200 con 0x5618c753fc90 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.706+0000 7f81b6ffd640 10 client.16755 handle_command_reply: tid=0 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.706+0000 7f81e061f740 1 client.16755 shutdown 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 msgr2=0x5618c74caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009900 tx=0x7f81cc02ab90 comp rx=0 tx=0).stop 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 shutdown_connections 2024-04-25T21:20:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5618c753fc90 0x5618c75420b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618c74c4800 0x5618c753dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5618c74b9f40 0x5618c74caf10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 --2- 172.21.15.37:0/3244103011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5618c74b9620 0x5618c74ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 >> 172.21.15.37:0/3244103011 conn(0x5618c74b5b30 msgr2=0x5618c74cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81b6ffd640 0 client.16755 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 shutdown_connections 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 1 -- 172.21.15.37:0/3244103011 wait complete. 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.707+0000 7f81e061f740 20 client.16755 trim_cache size 0 max 16384 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:20:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 msgr2=0x7f81c00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 0x7f81c00a6300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f81d0005fd0 tx=0x7f81d0009d70 comp rx=0 tx=0).stop 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 msgr2=0x7f81d8198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f81c800d950 tx=0x7f81c800de10 comp rx=0 tx=0).stop 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 shutdown_connections 2024-04-25T21:20:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81d819cf20 0x7f81d819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f81c00a3e40 0x7f81c00a6300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d8107270 0x7f81d8198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 --2- 172.21.15.37:0/3898981885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81d81021a0 0x7f81d8198660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 >> 172.21.15.37:0/3898981885 conn(0x7f81d80793e0 msgr2=0x7f81d80ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 shutdown_connections 2024-04-25T21:20:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:20.708+0000 7f81dd915640 1 -- 172.21.15.37:0/3898981885 wait complete. 2024-04-25T21:20:21.873 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:21 smithi037 ceph-mon[29195]: pgmap v371: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:21 smithi177 ceph-mon[33043]: pgmap v371: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:21 smithi098 ceph-mon[32569]: pgmap v371: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:22.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 -- 172.21.15.37:0/2054267708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802006a750 msgr2=0x7f802006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:22.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 --2- 172.21.15.37:0/2054267708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802006a750 0x7f802006abb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8010009990 tx=0x7f8010009ca0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 -- 172.21.15.37:0/2054267708 shutdown_connections 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 --2- 172.21.15.37:0/2054267708 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 0x7f802010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 --2- 172.21.15.37:0/2054267708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8020108a30 0x7f8020108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 --2- 172.21.15.37:0/2054267708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802006a750 0x7f802006abb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 -- 172.21.15.37:0/2054267708 >> 172.21.15.37:0/2054267708 conn(0x7f80200788f0 msgr2=0x7f8020078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 -- 172.21.15.37:0/2054267708 shutdown_connections 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.262+0000 7f80277a9700 1 -- 172.21.15.37:0/2054267708 wait complete. 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 Processor -- start 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 -- start start 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802006a750 0x7f8020199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:22.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 0x7f80201996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8020112c20 con 0x7f8020108a30 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8020112aa0 con 0x7f802006b0f0 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f80277a9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8020112da0 con 0x7f802006a750 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f8025d46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f8025d46700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57956/0 (socket says 172.21.15.37:57956) 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f8025d46700 1 -- 172.21.15.37:0/1765530526 learned_addr learned my addr 172.21.15.37:0/1765530526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f8025545700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802006a750 0x7f8020199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.263+0000 7f8024d44700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 0x7f80201996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:22.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 -- 172.21.15.37:0/1765530526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802006a750 msgr2=0x7f8020199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802006a750 0x7f8020199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 -- 172.21.15.37:0/1765530526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 msgr2=0x7f80201996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 0x7f80201996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 -- 172.21.15.37:0/1765530526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802019e1a0 con 0x7f8020108a30 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f8025d46700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f801800ada0 tx=0x7f801800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017020 con 0x7f8020108a30 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f801800cd80 con 0x7f8020108a30 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018005260 con 0x7f8020108a30 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802019e490 con 0x7f8020108a30 2024-04-25T21:20:22.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.264+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a5d10 con 0x7f8020108a30 2024-04-25T21:20:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.265+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8020069730 con 0x7f8020108a30 2024-04-25T21:20:22.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.266+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f801800c8d0 con 0x7f8020108a30 2024-04-25T21:20:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.267+0000 7f80167fc700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 0x7f800c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.267+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f80180cffa0 con 0x7f8020108a30 2024-04-25T21:20:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.267+0000 7f8025545700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 0x7f800c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:22.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.267+0000 7f8025545700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 0x7f800c0a6270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f801c0045b0 tx=0x7f801c00b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:22.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.272+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8018021020 con 0x7f8020108a30 2024-04-25T21:20:22.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.526+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802002df80 con 0x7f8020108a30 2024-04-25T21:20:22.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.526+0000 7f80167fc700 1 -- 172.21.15.37:0/1765530526 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f801809c670 con 0x7f8020108a30 2024-04-25T21:20:22.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:20:22.527 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 msgr2=0x7f800c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 0x7f800c0a6270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f801c0045b0 tx=0x7f801c00b3c0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 msgr2=0x7f802019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f801800ada0 tx=0x7f801800c330 comp rx=0 tx=0).stop 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 shutdown_connections 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f800c0a3db0 0x7f800c0a6270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8020108a30 0x7f802019da60 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802006a750 0x7f8020199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 --2- 172.21.15.37:0/1765530526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f802006b0f0 0x7f80201996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 >> 172.21.15.37:0/1765530526 conn(0x7f80200788f0 msgr2=0x7f80200ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 shutdown_connections 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:22.529+0000 7f80277a9700 1 -- 172.21.15.37:0/1765530526 wait complete. 2024-04-25T21:20:22.530 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:20:22.826 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:22 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1765530526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:20:23.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:22 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1765530526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:20:23.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:22 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1765530526' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:20:23.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 8fda637f --release --await' 2024-04-25T21:20:23.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 8fda637f --release --await 2024-04-25T21:20:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:20:23.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:23 smithi037 ceph-mon[29195]: pgmap v372: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:23 smithi177 ceph-mon[33043]: pgmap v372: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:24.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:23 smithi098 ceph-mon[32569]: pgmap v372: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:20:25.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/2644156041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0101960 msgr2=0x7fe7c0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/2644156041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0101960 0x7fe7c0101de0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009990 tx=0x7fe7a8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/2644156041 shutdown_connections 2024-04-25T21:20:25.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/2644156041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0102320 0x7fe7c010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/2644156041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0107960 0x7fe7c0107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/2644156041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0101960 0x7fe7c0101de0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/2644156041 >> 172.21.15.37:0/2644156041 conn(0x7fe7c00789e0 msgr2=0x7fe7c0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/2644156041 shutdown_connections 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.187+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/2644156041 wait complete. 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 Processor -- start 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 -- start start 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 0x7fe7c00fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0102320 0x7fe7c00feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0111ba0 con 0x7fe7c0107960 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0111a20 con 0x7fe7c0102320 2024-04-25T21:20:25.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7c5b1e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c0111d20 con 0x7fe7c0101960 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57964/0 (socket says 172.21.15.37:57964) 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.188+0000 7fe7bffff700 1 -- 172.21.15.37:0/1536039171 learned_addr learned my addr 172.21.15.37:0/1536039171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 0x7fe7c00fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7beffd700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0102320 0x7fe7c00feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 -- 172.21.15.37:0/1536039171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 msgr2=0x7fe7c00fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 0x7fe7c00fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 -- 172.21.15.37:0/1536039171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0102320 msgr2=0x7fe7c00feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0102320 0x7fe7c00feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 -- 172.21.15.37:0/1536039171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01a76f0 con 0x7fe7c0107960 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 0x7fe7c00fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bffff700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c970 tx=0x7fe7b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bcff9700 1 -- 172.21.15.37:0/1536039171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ce60 con 0x7fe7c0107960 2024-04-25T21:20:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bcff9700 1 -- 172.21.15.37:0/1536039171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b8004830 con 0x7fe7c0107960 2024-04-25T21:20:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01a79e0 con 0x7fe7c0107960 2024-04-25T21:20:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7bcff9700 1 -- 172.21.15.37:0/1536039171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80054c0 con 0x7fe7c0107960 2024-04-25T21:20:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.189+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a7ef0 con 0x7fe7c0107960 2024-04-25T21:20:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.192+0000 7fe7bcff9700 1 -- 172.21.15.37:0/1536039171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7b80049e0 con 0x7fe7c0107960 2024-04-25T21:20:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.193+0000 7fe7bcff9700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 0x7fe7ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.193+0000 7fe7bcff9700 1 -- 172.21.15.37:0/1536039171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fe7b80d0660 con 0x7fe7c0107960 2024-04-25T21:20:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.193+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 0x7fe7ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.193+0000 7fe7d03a7b80 1 Processor -- start 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 -- start start 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 0x7fe7ac0a6270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009fa0 tx=0x7fe7b0009570 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72345ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c723461e0 0x559c723485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c7228de30 con 0x559c7233a9f0 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c72234a60 con 0x559c7233b310 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559c72289df0 con 0x559c723461e0 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57976/0 (socket says 172.21.15.37:57976) 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2433333442 learned_addr learned my addr 172.21.15.37:0/2433333442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7beffd700 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72345ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2433333442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c723461e0 msgr2=0x559c723485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c723461e0 0x559c723485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2433333442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 msgr2=0x559c72345ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72345ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.194+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2433333442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c721db9d0 con 0x559c7233a9f0 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000cf30 tx=0x7fe7b0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec8acb81665bea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7b5ffb700 1 -- 172.21.15.37:0/2433333442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0028070 con 0x559c7233a9f0 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7b5ffb700 1 -- 172.21.15.37:0/2433333442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0018bc0 con 0x559c7233a9f0 2024-04-25T21:20:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7b5ffb700 1 -- 172.21.15.37:0/2433333442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00163a0 con 0x559c7233a9f0 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2433333442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 msgr2=0x559c7233add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.195+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000cf30 tx=0x7fe7b0011000 comp rx=0 tx=0).stop 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2433333442 shutdown_connections 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c723461e0 0x559c723485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233a9f0 0x559c7233add0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2433333442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72345ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2433333442 >> 172.21.15.37:0/2433333442 conn(0x559c72268790 msgr2=0x559c72336ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2433333442 shutdown_connections 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2433333442 wait complete. 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 -- start start 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.196+0000 7fe7d03a7b80 1 Processor -- start 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c723459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72342d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c72348bd0 con 0x559c723461e0 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559c7222ccd0 con 0x559c7233b310 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x559c72348d50 con 0x559c7233a9f0 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7d03a7b80 10 client.0 _ll_register_callbacks cb 0x559c72226a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57984/0 (socket says 172.21.15.37:57984) 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 -- 172.21.15.37:0/2397191685 learned_addr learned my addr 172.21.15.37:0/2397191685 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c723459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7beffd700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72342d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 -- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 msgr2=0x559c723459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c723459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 -- 172.21.15.37:0/2397191685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 msgr2=0x559c72342d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72342d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 -- 172.21.15.37:0/2397191685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c72245370 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe7bffff700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ce3b0 tx=0x7fe7b809c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55be60fe34aba39e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.197+0000 7fe79effd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b801d040 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fe7b80ce030 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b0018bc0 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7b8098040 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 10 client.16773 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c723535b0 con 0x559c723461e0 2024-04-25T21:20:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0018bc0 con 0x559c723461e0 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7b809c860 con 0x559c723461e0 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 10 client.16773 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 20 client.16773 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c72355bf0 0x559c72357fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 4 client.16773 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.198+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c7229b620 con 0x559c72355bf0 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.199+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c72355bf0 0x559c72357fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.199+0000 7fe79effd700 10 client.16773 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.199+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c72355bf0 0x559c72357fe0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6fdbbe13c7f7c94 server_cookie=8e3ac2522e6e7f23 in_seq=0 out_seq=0 2024-04-25T21:20:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe79effd700 1 -- 172.21.15.37:0/2397191685 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c7229b620 con 0x559c72355bf0 2024-04-25T21:20:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe79effd700 10 client.16773 handle_command_reply: tid=0 2024-04-25T21:20:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 client.16773 shutdown 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 msgr2=0x559c72343250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ce3b0 tx=0x7fe7b809c4e0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 shutdown_connections 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c72355bf0 0x559c72357fe0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c723461e0 0x559c72343250 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c723459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2397191685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c7233b310 0x559c72342d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 >> 172.21.15.37:0/2397191685 conn(0x559c72268790 msgr2=0x559c72336b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe79effd700 0 client.16773 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 shutdown_connections 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2397191685 wait complete. 2024-04-25T21:20:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.200+0000 7fe7d03a7b80 20 client.16773 trim_cache size 0 max 16384 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.220+0000 7fe7d03a7b80 1 Processor -- start 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.220+0000 7fe7d03a7b80 1 -- start start 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c7233ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01af5c0 con 0x559c7233b310 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b1100 con 0x559c723461e0 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01afc90 con 0x559c7233a9f0 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57990/0 (socket says 172.21.15.37:57990) 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7beffd700 1 -- 172.21.15.37:0/2075695980 learned_addr learned my addr 172.21.15.37:0/2075695980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:25.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bffff700 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c7233ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2075695980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 msgr2=0x559c7233ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c7233ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2075695980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 msgr2=0x559c7234e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 -- 172.21.15.37:0/2075695980 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c72338370 con 0x559c7233a9f0 2024-04-25T21:20:25.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dfd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0004b10 tx=0x7fe7b0004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ae880c2c143aea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.221+0000 7fe7b57fa700 1 -- 172.21.15.37:0/2075695980 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0036070 con 0x559c7233a9f0 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7b57fa700 1 -- 172.21.15.37:0/2075695980 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b00237e0 con 0x559c7233a9f0 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7b57fa700 1 -- 172.21.15.37:0/2075695980 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0028040 con 0x559c7233a9f0 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2075695980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 msgr2=0x559c7234dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dfd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0004b10 tx=0x7fe7b0004be0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2075695980 shutdown_connections 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c7233ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dfd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/2075695980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2075695980 >> 172.21.15.37:0/2075695980 conn(0x559c72268790 msgr2=0x559c72344dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2075695980 shutdown_connections 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.222+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/2075695980 wait complete. 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 -- start start 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 Processor -- start 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c723b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b0a30 con 0x559c7233b310 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01b0360 con 0x559c723461e0 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c01ad580 con 0x559c7233a9f0 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7d03a7b80 10 client.0 _ll_register_callbacks cb 0x559c72226a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7beffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58006/0 (socket says 172.21.15.37:58006) 2024-04-25T21:20:25.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7beffd700 1 -- 172.21.15.37:0/3178317206 learned_addr learned my addr 172.21.15.37:0/3178317206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.223+0000 7fe7bffff700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c723b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 -- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 msgr2=0x559c7234dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 -- 172.21.15.37:0/3178317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 msgr2=0x559c723b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c723b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 -- 172.21.15.37:0/3178317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c723b3e90 con 0x559c7233b310 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7beffd700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098e0 tx=0x7fe7a800bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737cc18f391e20b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe79effd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8003c70 con 0x559c7233b310 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fe7a8005070 con 0x559c7233b310 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b00237e0 con 0x559c7233b310 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a8033050 con 0x559c7233b310 2024-04-25T21:20:25.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7d03a7b80 10 client.16779 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c723bc440 con 0x559c7233b310 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.224+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b00237e0 con 0x559c7233b310 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7a800f870 con 0x559c7233b310 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 10 client.16779 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 10 client.16779 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 20 client.16779 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c723bcbd0 0x559c723beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 4 client.16779 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8fda637f", "release": true, "await": true}] 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8fda637f", "release": true, "await": true}) -- 0x559c722956c0 con 0x559c723bcbd0 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c723bcbd0 0x559c723beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe7bf7fe700 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c723bcbd0 0x559c723beff0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c77f9e1e78d0ad71 server_cookie=70f6d648f7030838 in_seq=0 out_seq=0 2024-04-25T21:20:25.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.225+0000 7fe79effd700 10 client.16779 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:25.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe79effd700 1 -- 172.21.15.37:0/3178317206 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559c722956c0 con 0x559c723bcbd0 2024-04-25T21:20:25.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe79effd700 10 client.16779 handle_command_reply: tid=0 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 client.16779 shutdown 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 msgr2=0x559c7234e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098e0 tx=0x7fe7a800bf20 comp rx=0 tx=0).stop 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 shutdown_connections 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x559c723bcbd0 0x559c723beff0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559c723461e0 0x559c723b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559c7233a9f0 0x559c7234dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 --2- 172.21.15.37:0/3178317206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c7233b310 0x559c7234e100 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 >> 172.21.15.37:0/3178317206 conn(0x559c72268790 msgr2=0x559c72350ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe79effd700 0 client.16779 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 shutdown_connections 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 1 -- 172.21.15.37:0/3178317206 wait complete. 2024-04-25T21:20:25.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7d03a7b80 20 client.16779 trim_cache size 0 max 16384 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 32, 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "8fda637f": { 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 32, 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:20:25.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 88.1, 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:20:25.232 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:20:25.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 msgr2=0x7fe7ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 0x7fe7ac0a6270 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009fa0 tx=0x7fe7b0009570 comp rx=0 tx=0).stop 2024-04-25T21:20:25.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 msgr2=0x7fe7c00ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:25.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c970 tx=0x7fe7b800cc80 comp rx=0 tx=0).stop 2024-04-25T21:20:25.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 shutdown_connections 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7ac0a3db0 0x7fe7ac0a6270 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7c0107960 0x7fe7c00ff410 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7c0101960 0x7fe7c00fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 --2- 172.21.15.37:0/1536039171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7c0102320 0x7fe7c00feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 >> 172.21.15.37:0/1536039171 conn(0x7fe7c00789e0 msgr2=0x7fe7c006b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 shutdown_connections 2024-04-25T21:20:25.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:20:25.230+0000 7fe7c5b1e700 1 -- 172.21.15.37:0/1536039171 wait complete. 2024-04-25T21:20:25.945 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8fda637f', seconds elapsed: 40.0 2024-04-25T21:20:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:25 smithi037 ceph-mon[29195]: pgmap v373: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T21:20:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:25 smithi177 ceph-mon[33043]: pgmap v373: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T21:20:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:25 smithi098 ceph-mon[32569]: pgmap v373: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2024-04-25T21:20:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:27 smithi037 ceph-mon[29195]: pgmap v374: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-25T21:20:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:27 smithi177 ceph-mon[33043]: pgmap v374: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-25T21:20:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:27 smithi098 ceph-mon[32569]: pgmap v374: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-25T21:20:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:29 smithi037 ceph-mon[29195]: pgmap v375: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:20:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:29 smithi177 ceph-mon[33043]: pgmap v375: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:20:30.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:29 smithi098 ceph-mon[32569]: pgmap v375: 97 pgs: 97 active+clean; 3.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:20:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:31 smithi037 ceph-mon[29195]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:20:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:31 smithi177 ceph-mon[33043]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:20:32.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:31 smithi098 ceph-mon[32569]: pgmap v376: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- 172.21.15.37:0/4050153191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 msgr2=0x7fa8f810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 --2- 172.21.15.37:0/4050153191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f810ee60 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009ca0 tx=0x7fa8f002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- 172.21.15.37:0/4050153191 shutdown_connections 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 --2- 172.21.15.37:0/4050153191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f810ee60 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 --2- 172.21.15.37:0/4050153191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f8102b70 0x7fa8f8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 --2- 172.21.15.37:0/4050153191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- 172.21.15.37:0/4050153191 >> 172.21.15.37:0/4050153191 conn(0x7fa8f80fda20 msgr2=0x7fa8f80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- 172.21.15.37:0/4050153191 shutdown_connections 2024-04-25T21:20:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- 172.21.15.37:0/4050153191 wait complete. 2024-04-25T21:20:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 Processor -- start 2024-04-25T21:20:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.811+0000 7fa8fe810640 1 -- start start 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f819a680 0x7fa8f819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8112230 con 0x7fa8f8107290 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81120b0 con 0x7fa8f819a680 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fe810640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81123b0 con 0x7fa8f81021c0 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:46920/0 (socket says 172.21.15.37:46920) 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59586/0 (socket says 172.21.15.37:59586) 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f7fff640 1 -- 172.21.15.37:0/1053806988 learned_addr learned my addr 172.21.15.37:0/1053806988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f819a680 0x7fa8f819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 -- 172.21.15.37:0/1053806988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 msgr2=0x7fa8f81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 -- 172.21.15.37:0/1053806988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f819a680 msgr2=0x7fa8f819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f819a680 0x7fa8f819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.812+0000 7fa8f77fe640 1 -- 172.21.15.37:0/1053806988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f819b280 con 0x7fa8f8107290 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e9c0 tx=0x7fa8ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8f57fa640 1 -- 172.21.15.37:0/1053806988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00ccc0 con 0x7fa8f8107290 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f81a7360 con 0x7fa8f8107290 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8f57fa640 1 -- 172.21.15.37:0/1053806988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec0108e0 con 0x7fa8f8107290 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8f57fa640 1 -- 172.21.15.37:0/1053806988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0187a0 con 0x7fa8f8107290 2024-04-25T21:20:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.813+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f81a77c0 con 0x7fa8f8107290 2024-04-25T21:20:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.815+0000 7fa8f57fa640 1 -- 172.21.15.37:0/1053806988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8ec010430 con 0x7fa8f8107290 2024-04-25T21:20:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.815+0000 7fa901601740 1 Processor -- start 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 -- start start 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca569a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca5742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca4ce970 con 0x55c4ca574800 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca46b4a0 con 0x55c4ca569f40 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa901601740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca4ca930 con 0x55c4ca569620 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f57fa640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 0x7fa8e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59602/0 (socket says 172.21.15.37:59602) 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 -- 172.21.15.37:0/346618455 learned_addr learned my addr 172.21.15.37:0/346618455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 0x7fa8e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca5742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f57fa640 1 -- 172.21.15.37:0/1053806988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fa8ec0d0a80 con 0x7fa8f8107290 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca569a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 -- 172.21.15.37:0/346618455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 msgr2=0x55c4ca569a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca569a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 -- 172.21.15.37:0/346618455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 msgr2=0x55c4ca5742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca5742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8f77fe640 1 -- 172.21.15.37:0/346618455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ca1e1210 con 0x55c4ca574800 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.816+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca569a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec09c7d0 tx=0x7fa8ec00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1641706debd86e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 0x7fa8e00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0006010 tx=0x7fa8f0009360 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa8de7fc640 1 -- 172.21.15.37:0/346618455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec026040 con 0x55c4ca574800 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa8de7fc640 1 -- 172.21.15.37:0/346618455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec065e50 con 0x55c4ca574800 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa8de7fc640 1 -- 172.21.15.37:0/346618455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0ce450 con 0x55c4ca574800 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa901601740 1 -- 172.21.15.37:0/346618455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 msgr2=0x55c4ca576bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.817+0000 7fa901601740 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec09c7d0 tx=0x7fa8ec00ce60 comp rx=0 tx=0).stop 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- 172.21.15.37:0/346618455 shutdown_connections 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca576bf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca5742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- 172.21.15.37:0/346618455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca569a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- 172.21.15.37:0/346618455 >> 172.21.15.37:0/346618455 conn(0x55c4ca565b30 msgr2=0x55c4ca565f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- 172.21.15.37:0/346618455 shutdown_connections 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- 172.21.15.37:0/346618455 wait complete. 2024-04-25T21:20:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- start start 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 Processor -- start 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569620 0x55c4ca574190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569f40 0x55c4ca571480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca5773d0 con 0x55c4ca574800 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca4690b0 con 0x55c4ca569620 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.818+0000 7fa901601740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c4ca577550 con 0x55c4ca569f40 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa901601740 10 client.0 _ll_register_callbacks cb 0x55c4ca456d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59606/0 (socket says 172.21.15.37:59606) 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 -- 172.21.15.37:0/3320332787 learned_addr learned my addr 172.21.15.37:0/3320332787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569620 0x55c4ca574190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569f40 0x55c4ca571480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 -- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569f40 msgr2=0x55c4ca571480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569f40 0x55c4ca571480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 -- 172.21.15.37:0/3320332787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569620 msgr2=0x55c4ca574190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569620 0x55c4ca574190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 -- 172.21.15.37:0/3320332787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ca2ad620 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009430 tx=0x7fa8f002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cdac12e08546d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0050020 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fa8f0003b80 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.819+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec065e50 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8f005b050 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 10 client.16791 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4ca583320 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec065e50 con 0x55c4ca574800 2024-04-25T21:20:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8f002f6f0 con 0x55c4ca574800 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 10 client.16791 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 20 client.16791 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca584830 0x55c4ca586c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 4 client.16791 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ca4dc390 con 0x55c4ca584830 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.820+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca584830 0x55c4ca586c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.821+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca584830 0x55c4ca586c20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b8951105a5300f server_cookie=ee8a1a231f4f3def in_seq=0 out_seq=0 2024-04-25T21:20:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.821+0000 7fa8d77fe640 10 client.16791 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.821+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3320332787 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4ca4dc390 con 0x55c4ca584830 2024-04-25T21:20:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa8d77fe640 10 client.16791 handle_command_reply: tid=0 2024-04-25T21:20:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 client.16791 shutdown 2024-04-25T21:20:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 msgr2=0x55c4ca571a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009430 tx=0x7fa8f002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 shutdown_connections 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca584830 0x55c4ca586c20 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca574800 0x55c4ca571a00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569f40 0x55c4ca571480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa8d77fe640 0 client.16791 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 --2- 172.21.15.37:0/3320332787 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569620 0x55c4ca574190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 >> 172.21.15.37:0/3320332787 conn(0x55c4ca565b30 msgr2=0x55c4ca56b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 shutdown_connections 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 1 -- 172.21.15.37:0/3320332787 wait complete. 2024-04-25T21:20:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.822+0000 7fa901601740 20 client.16791 trim_cache size 0 max 16384 2024-04-25T21:20:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 Processor -- start 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 -- start start 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca580290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 0x55c4ca58c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81afd20 con 0x55c4ca569f40 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81b1860 con 0x55c4ca574800 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa901601740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81b03f0 con 0x55c4ca569620 2024-04-25T21:20:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59622/0 (socket says 172.21.15.37:59622) 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa8f77fe640 1 -- 172.21.15.37:0/729023019 learned_addr learned my addr 172.21.15.37:0/729023019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca580290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.837+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 0x55c4ca58c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 -- 172.21.15.37:0/729023019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 msgr2=0x55c4ca580290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca580290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 -- 172.21.15.37:0/729023019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 msgr2=0x55c4ca58c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 0x55c4ca58c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 -- 172.21.15.37:0/729023019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ca566cc0 con 0x55c4ca569f40 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 0x55c4ca58c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0ce7d0 tx=0x7fa8ec009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f799e38951fabe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8ddffb640 1 -- 172.21.15.37:0/729023019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec065030 con 0x55c4ca569f40 2024-04-25T21:20:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8ddffb640 1 -- 172.21.15.37:0/729023019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec0a0dd0 con 0x55c4ca569f40 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa8ddffb640 1 -- 172.21.15.37:0/729023019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec022020 con 0x55c4ca569f40 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa901601740 1 -- 172.21.15.37:0/729023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 msgr2=0x55c4ca5703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.838+0000 7fa901601740 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0ce7d0 tx=0x7fa8ec009700 comp rx=0 tx=0).stop 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- 172.21.15.37:0/729023019 shutdown_connections 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca574800 0x55c4ca58c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569f40 0x55c4ca5703e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- 172.21.15.37:0/729023019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca569620 0x55c4ca580290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- 172.21.15.37:0/729023019 >> 172.21.15.37:0/729023019 conn(0x55c4ca565b30 msgr2=0x55c4ca564cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- 172.21.15.37:0/729023019 shutdown_connections 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- 172.21.15.37:0/729023019 wait complete. 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- start start 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 Processor -- start 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca57a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 0x55c4ca57abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81b1190 con 0x55c4ca569620 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81b0ac0 con 0x55c4ca569f40 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f81add20 con 0x55c4ca574800 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.839+0000 7fa901601740 10 client.0 _ll_register_callbacks cb 0x7fa8f0031ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59626/0 (socket says 172.21.15.37:59626) 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 -- 172.21.15.37:0/3490646416 learned_addr learned my addr 172.21.15.37:0/3490646416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca57a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 0x55c4ca57abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 -- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 msgr2=0x55c4ca57abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 0x55c4ca57abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 -- 172.21.15.37:0/3490646416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 msgr2=0x55c4ca57a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca57a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 -- 172.21.15.37:0/3490646416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ca5e9d70 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8fcdad640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 0x55c4ca57abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8f7fff640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8004650 tx=0x7fa8e801f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e185a61294a13250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8005020 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7fa8e80051c0 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4ca4d6200 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8e802d050 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa901601740 10 client.16797 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.840+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4ca5eb060 con 0x55c4ca569620 2024-04-25T21:20:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c4ca4d6200 con 0x55c4ca569620 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa8e8003680 con 0x55c4ca569620 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 10 client.16797 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 20 client.16797 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca5eb7b0 0x55c4ca5edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 4 client.16797 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4ca4525f0 con 0x55c4ca5eb7b0 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca5eb7b0 0x55c4ca5edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa8f77fe640 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca5eb7b0 0x55c4ca5edbd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c89e680b0d0b128 server_cookie=93d4c17f1a864ee4 in_seq=0 out_seq=0 2024-04-25T21:20:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.841+0000 7fa8d77fe640 10 client.16797 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa8d77fe640 1 -- 172.21.15.37:0/3490646416 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4ca4525f0 con 0x55c4ca5eb7b0 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa8d77fe640 10 client.16797 handle_command_reply: tid=0 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 client.16797 shutdown 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 msgr2=0x55c4ca57a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8004650 tx=0x7fa8e801f720 comp rx=0 tx=0).stop 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 shutdown_connections 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c4ca5eb7b0 0x55c4ca5edbd0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c4ca574800 0x55c4ca57abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa8d77fe640 0 client.16797 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c4ca569f40 0x55c4ca57a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 --2- 172.21.15.37:0/3490646416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c4ca569620 0x55c4ca57a170 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 >> 172.21.15.37:0/3490646416 conn(0x55c4ca565b30 msgr2=0x55c4ca57f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 shutdown_connections 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.855+0000 7fa901601740 1 -- 172.21.15.37:0/3490646416 wait complete. 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa901601740 20 client.16797 trim_cache size 0 max 16384 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:20:33.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 msgr2=0x7fa8e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 0x7fa8e00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0006010 tx=0x7fa8f0009360 comp rx=0 tx=0).stop 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 msgr2=0x7fa8f81a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e9c0 tx=0x7fa8ec00ee80 comp rx=0 tx=0).stop 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 shutdown_connections 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f819a680 0x7fa8f819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8f8107290 0x7fa8f81a0b00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fa8e00a3dc0 0x7fa8e00a6280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 --2- 172.21.15.37:0/1053806988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8f81021c0 0x7fa8f81a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 >> 172.21.15.37:0/1053806988 conn(0x7fa8f80fda20 msgr2=0x7fa8f810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 shutdown_connections 2024-04-25T21:20:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:33.856+0000 7fa8fe810640 1 -- 172.21.15.37:0/1053806988 wait complete. 2024-04-25T21:20:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:33 smithi037 ceph-mon[29195]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T21:20:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:33 smithi177 ceph-mon[33043]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T21:20:34.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:33 smithi098 ceph-mon[32569]: pgmap v377: 97 pgs: 97 active+clean; 3.9 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-04-25T21:20:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:35 smithi177 ceph-mon[33043]: pgmap v378: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-25T21:20:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:35 smithi098 ceph-mon[32569]: pgmap v378: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-25T21:20:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:35 smithi037 ceph-mon[29195]: pgmap v378: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-25T21:20:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:37 smithi177 ceph-mon[33043]: pgmap v379: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:37 smithi098 ceph-mon[32569]: pgmap v379: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:37 smithi037 ceph-mon[29195]: pgmap v379: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:39 smithi177 ceph-mon[33043]: pgmap v380: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:40.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:39 smithi098 ceph-mon[32569]: pgmap v380: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:39 smithi037 ceph-mon[29195]: pgmap v380: 97 pgs: 97 active+clean; 4.2 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 14 op/s 2024-04-25T21:20:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:41 smithi177 ceph-mon[33043]: pgmap v381: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T21:20:42.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:41 smithi098 ceph-mon[32569]: pgmap v381: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T21:20:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:41 smithi037 ceph-mon[29195]: pgmap v381: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-25T21:20:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:43 smithi177 ceph-mon[33043]: pgmap v382: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:20:44.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:43 smithi098 ceph-mon[32569]: pgmap v382: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:20:44.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:43 smithi037 ceph-mon[29195]: pgmap v382: 97 pgs: 97 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:20:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:45 smithi177 ceph-mon[33043]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:20:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:45 smithi098 ceph-mon[32569]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:20:46.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:45 smithi037 ceph-mon[29195]: pgmap v383: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- 172.21.15.37:0/3476346695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 msgr2=0x7f690010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 --2- 172.21.15.37:0/3476346695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690010e9c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009ca0 tx=0x7f68f802a9e0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- 172.21.15.37:0/3476346695 shutdown_connections 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 --2- 172.21.15.37:0/3476346695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690010e9c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 --2- 172.21.15.37:0/3476346695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6900103b00 0x7f690010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 --2- 172.21.15.37:0/3476346695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f69001031e0 0x7f69001035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- 172.21.15.37:0/3476346695 >> 172.21.15.37:0/3476346695 conn(0x7f6900077fd0 msgr2=0x7f69000783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- 172.21.15.37:0/3476346695 shutdown_connections 2024-04-25T21:20:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- 172.21.15.37:0/3476346695 wait complete. 2024-04-25T21:20:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 Processor -- start 2024-04-25T21:20:46.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.947+0000 7f6906010640 1 -- start start 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69001031e0 0x7f69001986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 0x7f6900198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6900112200 con 0x7f690010c520 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6900112080 con 0x7f6900103b00 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f6906010640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6900112380 con 0x7f69001031e0 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68feffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 0x7f6900198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37100/0 (socket says 172.21.15.37:37100) 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 -- 172.21.15.37:0/3791296369 learned_addr learned my addr 172.21.15.37:0/3791296369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68feffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 0x7f6900198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:52674/0 (socket says 172.21.15.37:52674) 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69001031e0 0x7f69001986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 -- 172.21.15.37:0/3791296369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69001031e0 msgr2=0x7f69001986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69001031e0 0x7f69001986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 -- 172.21.15.37:0/3791296369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 msgr2=0x7f6900198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 0x7f6900198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 -- 172.21.15.37:0/3791296369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690019d680 con 0x7f690010c520 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.948+0000 7f68fffff640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f68f802f040 tx=0x7f68f802fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.949+0000 7f68fcff9640 1 -- 172.21.15.37:0/3791296369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8033040 con 0x7f690010c520 2024-04-25T21:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.949+0000 7f68fcff9640 1 -- 172.21.15.37:0/3791296369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f802e820 con 0x7f690010c520 2024-04-25T21:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.949+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690019d910 con 0x7f690010c520 2024-04-25T21:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.949+0000 7f68fcff9640 1 -- 172.21.15.37:0/3791296369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8043750 con 0x7f690010c520 2024-04-25T21:20:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.949+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69001a5200 con 0x7f690010c520 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f68fcff9640 1 -- 172.21.15.37:0/3791296369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68f8036070 con 0x7f690010c520 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 Processor -- start 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 -- start start 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae25a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae32bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55957ad8a970 con 0x55957ae25f40 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55957ad274a0 con 0x55957ae25620 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f6908e03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55957ad86930 con 0x55957ae30800 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.951+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37114/0 (socket says 172.21.15.37:37114) 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 -- 172.21.15.37:0/2850160683 learned_addr learned my addr 172.21.15.37:0/2850160683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68feffd640 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae32bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68feffd640 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae25a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 -- 172.21.15.37:0/2850160683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 msgr2=0x55957ae32bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae32bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 -- 172.21.15.37:0/2850160683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 msgr2=0x55957ae25a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae25a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 -- 172.21.15.37:0/2850160683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957aa9d210 con 0x55957ae25f40 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68fcff9640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 0x7f68e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f68f000bcb0 tx=0x7f68f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6edd17a92f305bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68feffd640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 0x7f68e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68fcff9640 1 -- 172.21.15.37:0/3791296369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f68f80f1650 con 0x7f690010c520 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68e5ffb640 1 -- 172.21.15.37:0/2850160683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0015070 con 0x55957ae25f40 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68e5ffb640 1 -- 172.21.15.37:0/2850160683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f001a910 con 0x55957ae25f40 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.952+0000 7f68e5ffb640 1 -- 172.21.15.37:0/2850160683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0011920 con 0x55957ae25f40 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 -- 172.21.15.37:0/2850160683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 msgr2=0x55957ae302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f68f000bcb0 tx=0x7f68f000f730 comp rx=0 tx=0).stop 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f68feffd640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 0x7f68e80a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55957ae22f50 tx=0x7f68f400b390 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 -- 172.21.15.37:0/2850160683 shutdown_connections 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae32bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae302c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 --2- 172.21.15.37:0/2850160683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae25a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 -- 172.21.15.37:0/2850160683 >> 172.21.15.37:0/2850160683 conn(0x55957ae21b30 msgr2=0x55957ae21f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 -- 172.21.15.37:0/2850160683 shutdown_connections 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.953+0000 7f6908e03740 1 -- 172.21.15.37:0/2850160683 wait complete. 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 -- start start 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 Processor -- start 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae30130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae2d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55957ae33430 con 0x55957ae25f40 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55957ad250b0 con 0x55957ae25620 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55957ae335b0 con 0x55957ae30800 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f6908e03740 10 client.0 _ll_register_callbacks cb 0x55957ad12d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f68feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae2d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f68feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae2d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37122/0 (socket says 172.21.15.37:37122) 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f68feffd640 1 -- 172.21.15.37:0/1304288057 learned_addr learned my addr 172.21.15.37:0/1304288057 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae30130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.954+0000 7f68fffff640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 -- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 msgr2=0x55957ae2d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae2d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 -- 172.21.15.37:0/1304288057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 msgr2=0x55957ae2d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae2d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 -- 172.21.15.37:0/1304288057 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957ab69620 con 0x55957ae25620 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae30130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55957ae26bc0 tx=0x7f68f000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdaaea7fc96a18be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68deffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0015070 con 0x55957ae25620 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f68f001a670 con 0x55957ae25620 2024-04-25T21:20:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f001a910 con 0x55957ae25620 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68f001aac0 con 0x55957ae25620 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f6908e03740 10 client.26105 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.955+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55957ae43790 con 0x55957ae25620 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f001a910 con 0x55957ae25620 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f68f0005340 con 0x55957ae25620 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 10 client.26105 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 10 client.26105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 20 client.26105 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae44b90 0x55957ae46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 4 client.26105 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957ad98390 con 0x55957ae44b90 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.956+0000 7f68feffd640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae44b90 0x55957ae46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f68deffd640 10 client.26105 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:46.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f68feffd640 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae44b90 0x55957ae46f80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225d946bd200901f server_cookie=b558e888eab419cf in_seq=0 out_seq=0 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f68deffd640 1 -- 172.21.15.37:0/1304288057 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55957ad98390 con 0x55957ae44b90 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f68deffd640 10 client.26105 handle_command_reply: tid=0 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f6908e03740 1 client.26105 shutdown 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 msgr2=0x55957ae30130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae30130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55957ae26bc0 tx=0x7f68f000bfd0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 shutdown_connections 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.957+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae44b90 0x55957ae46f80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae30800 0x55957ae2d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25f40 0x55957ae2d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 --2- 172.21.15.37:0/1304288057 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae30130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f68deffd640 0 client.26105 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 >> 172.21.15.37:0/1304288057 conn(0x55957ae21b30 msgr2=0x55957ae278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 shutdown_connections 2024-04-25T21:20:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 1 -- 172.21.15.37:0/1304288057 wait complete. 2024-04-25T21:20:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.958+0000 7f6908e03740 20 client.26105 trim_cache size 0 max 16384 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 Processor -- start 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 -- start start 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae25f40 0x55957ae4cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ab900 con 0x55957ae21b30 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ad440 con 0x55957ae25620 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f6908e03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69001abfd0 con 0x55957ae25f40 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37124/0 (socket says 172.21.15.37:37124) 2024-04-25T21:20:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 -- 172.21.15.37:0/3227140286 learned_addr learned my addr 172.21.15.37:0/3227140286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68feffd640 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68fffff640 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae25f40 0x55957ae4cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 -- 172.21.15.37:0/3227140286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae25f40 msgr2=0x55957ae4cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae25f40 0x55957ae4cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 -- 172.21.15.37:0/3227140286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 msgr2=0x55957ae4c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae4c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.973+0000 7f68ff7fe640 1 -- 172.21.15.37:0/3227140286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957ae22d60 con 0x55957ae21b30 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f68f000ff20 tx=0x7f68f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51db2a9cd02a02f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f68e57fa640 1 -- 172.21.15.37:0/3227140286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0015040 con 0x55957ae21b30 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f68e57fa640 1 -- 172.21.15.37:0/3227140286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f00049b0 con 0x55957ae21b30 2024-04-25T21:20:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f68e57fa640 1 -- 172.21.15.37:0/3227140286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0011aa0 con 0x55957ae21b30 2024-04-25T21:20:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 -- 172.21.15.37:0/3227140286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 msgr2=0x55957ae2fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f68f000ff20 tx=0x7f68f000b910 comp rx=0 tx=0).stop 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 -- 172.21.15.37:0/3227140286 shutdown_connections 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae25f40 0x55957ae4cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25620 0x55957ae4c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 --2- 172.21.15.37:0/3227140286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae21b30 0x55957ae2fc60 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.974+0000 7f6908e03740 1 -- 172.21.15.37:0/3227140286 >> 172.21.15.37:0/3227140286 conn(0x55957ae2dac0 msgr2=0x55957ae28af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- 172.21.15.37:0/3227140286 shutdown_connections 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- 172.21.15.37:0/3227140286 wait complete. 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- start start 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 Processor -- start 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae21b30 0x55957ae3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25f40 0x55957ae389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69001acd70 con 0x55957ae25620 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ac6a0 con 0x55957ae25f40 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f69001a9920 con 0x55957ae21b30 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f6908e03740 10 client.0 _ll_register_callbacks cb 0x7f68f000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f68feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f68feffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37134/0 (socket says 172.21.15.37:37134) 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f68feffd640 1 -- 172.21.15.37:0/1083057203 learned_addr learned my addr 172.21.15.37:0/1083057203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f68fffff640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25f40 0x55957ae389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.975+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae21b30 0x55957ae3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 -- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae21b30 msgr2=0x55957ae3d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae21b30 0x55957ae3d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 -- 172.21.15.37:0/1083057203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25f40 msgr2=0x55957ae389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:20:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25f40 0x55957ae389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 -- 172.21.15.37:0/1083057203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957ae39210 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68feffd640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a991a9568dc4f1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4030070 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f68f4003cd0 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f00049b0 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68f403c050 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f6908e03740 10 client.16815 fetch_fsmap learned FSMap version 27 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55957aeaaef0 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f00049b0 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.976+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f68f401d070 con 0x55957ae25620 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 10 client.16815 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:20:46.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 20 client.16815 populate_metadata read hostname 'smithi037' 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae30800 0x55957aead660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 4 client.16815 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55957ad92200 con 0x55957ae30800 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae30800 0x55957aead660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f68ff7fe640 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae30800 0x55957aead660 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2b05b7be5de13e server_cookie=da6dc3e84efe0b40 in_seq=0 out_seq=0 2024-04-25T21:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.977+0000 7f68deffd640 10 client.16815 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f68deffd640 1 -- 172.21.15.37:0/1083057203 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55957ad92200 con 0x55957ae30800 2024-04-25T21:20:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f68deffd640 10 client.16815 handle_command_reply: tid=0 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 client.16815 shutdown 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 msgr2=0x55957ae38480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f4004730 comp rx=0 tx=0).stop 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 shutdown_connections 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55957ae30800 0x55957aead660 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55957ae25f40 0x55957ae389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55957ae25620 0x55957ae38480 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 --2- 172.21.15.37:0/1083057203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55957ae21b30 0x55957ae3d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f68deffd640 0 client.16815 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 >> 172.21.15.37:0/1083057203 conn(0x55957ae2dac0 msgr2=0x55957ae3f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 shutdown_connections 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 1 -- 172.21.15.37:0/1083057203 wait complete. 2024-04-25T21:20:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.990+0000 7f6908e03740 20 client.16815 trim_cache size 0 max 16384 2024-04-25T21:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 msgr2=0x7f68e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 0x7f68e80a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x55957ae22f50 tx=0x7f68f400b390 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 msgr2=0x7f690019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f68f802f040 tx=0x7f68f802fbe0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 shutdown_connections 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f690010c520 0x7f690019cf50 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6900103b00 0x7f6900198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f68e80a3dc0 0x7f68e80a6280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 --2- 172.21.15.37:0/3791296369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69001031e0 0x7f69001986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 >> 172.21.15.37:0/3791296369 conn(0x7f6900077fd0 msgr2=0x7f69000ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 shutdown_connections 2024-04-25T21:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:20:46.991+0000 7f6906010640 1 -- 172.21.15.37:0/3791296369 wait complete. 2024-04-25T21:20:47.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:46 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:20:47.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:46 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:20:47.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:46 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:20:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:47 smithi177 ceph-mon[33043]: pgmap v384: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:20:48.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:20:48.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:20:48.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:47 smithi098 ceph-mon[32569]: pgmap v384: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:20:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:20:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:20:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:47 smithi037 ceph-mon[29195]: pgmap v384: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:20:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:20:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:20:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:49 smithi177 ceph-mon[33043]: pgmap v385: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:50.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:49 smithi098 ceph-mon[32569]: pgmap v385: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:50.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:49 smithi037 ceph-mon[29195]: pgmap v385: 97 pgs: 97 active+clean; 4.5 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-25T21:20:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:51 smithi177 ceph-mon[33043]: pgmap v386: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:51 smithi098 ceph-mon[32569]: pgmap v386: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:52.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:51 smithi037 ceph-mon[29195]: pgmap v386: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:53 smithi177 ceph-mon[33043]: pgmap v387: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:20:54.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:53 smithi098 ceph-mon[32569]: pgmap v387: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:20:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:53 smithi037 ceph-mon[29195]: pgmap v387: 97 pgs: 97 active+clean; 4.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:20:56.036 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:55 smithi037 ceph-mon[29195]: pgmap v388: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:55 smithi177 ceph-mon[33043]: pgmap v388: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:56.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:55 smithi098 ceph-mon[32569]: pgmap v388: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:20:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:57 smithi177 ceph-mon[33043]: pgmap v389: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:20:58.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:57 smithi098 ceph-mon[32569]: pgmap v389: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:20:58.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:57 smithi037 ceph-mon[29195]: pgmap v389: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:21:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:20:59 smithi177 ceph-mon[33043]: pgmap v390: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:21:00.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 -- 172.21.15.37:0/1730635435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c10c550 msgr2=0x7f4b8c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 --2- 172.21.15.37:0/1730635435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c10c550 0x7f4b8c10e9d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009ca0 tx=0x7f4b8402a950 comp rx=0 tx=0).stop 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 -- 172.21.15.37:0/1730635435 shutdown_connections 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 --2- 172.21.15.37:0/1730635435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c10c550 0x7f4b8c10e9d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 --2- 172.21.15.37:0/1730635435 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 --2- 172.21.15.37:0/1730635435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c103180 0x7f4b8c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 -- 172.21.15.37:0/1730635435 >> 172.21.15.37:0/1730635435 conn(0x7f4b8c077fd0 msgr2=0x7f4b8c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 -- 172.21.15.37:0/1730635435 shutdown_connections 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.091+0000 7f4b934dc640 1 -- 172.21.15.37:0/1730635435 wait complete. 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 Processor -- start 2024-04-25T21:21:00.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 -- start start 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103180 0x7f4b8c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c10c550 0x7f4b8c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1121c0 con 0x7f4b8c103aa0 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c112040 con 0x7f4b8c103180 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b934dc640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c112340 con 0x7f4b8c10c550 2024-04-25T21:21:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33996/0 (socket says 172.21.15.37:33996) 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.092+0000 7f4b90a77640 1 -- 172.21.15.37:0/3575278071 learned_addr learned my addr 172.21.15.37:0/3575278071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c10c550 0x7f4b8c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b91278640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103180 0x7f4b8c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 -- 172.21.15.37:0/3575278071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c10c550 msgr2=0x7f4b8c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c10c550 0x7f4b8c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 -- 172.21.15.37:0/3575278071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103180 msgr2=0x7f4b8c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103180 0x7f4b8c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 -- 172.21.15.37:0/3575278071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c19d650 con 0x7f4b8c103aa0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d950 tx=0x7f4b7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b827fc640 1 -- 172.21.15.37:0/3575278071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c017030 con 0x7f4b8c103aa0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b827fc640 1 -- 172.21.15.37:0/3575278071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c004020 con 0x7f4b8c103aa0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c19d940 con 0x7f4b8c103aa0 2024-04-25T21:21:00.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b827fc640 1 -- 172.21.15.37:0/3575278071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0047a0 con 0x7f4b8c103aa0 2024-04-25T21:21:00.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.093+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1a51d0 con 0x7f4b8c103aa0 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b827fc640 1 -- 172.21.15.37:0/3575278071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b7c009a30 con 0x7f4b8c103aa0 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 Processor -- start 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 -- start start 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e31fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e32a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c57e284970 con 0x55c57e32a800 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c57e2214a0 con 0x55c57e31f620 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.096+0000 7f4b961ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c57e280930 con 0x55c57e31ff40 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34008/0 (socket says 172.21.15.37:34008) 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 -- 172.21.15.37:0/893460667 learned_addr learned my addr 172.21.15.37:0/893460667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b91278640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e32a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b91a79640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e31fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b827fc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 0x7f4b780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 -- 172.21.15.37:0/893460667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 msgr2=0x55c57e32a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e32a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 -- 172.21.15.37:0/893460667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 msgr2=0x55c57e31fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e31fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 -- 172.21.15.37:0/893460667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57df97210 con 0x55c57e32a800 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b91278640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e32a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b91a79640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e31fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 0x7f4b780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b827fc640 1 -- 172.21.15.37:0/3575278071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f4b7c0d0100 con 0x7f4b8c103aa0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.097+0000 7f4b90a77640 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0cd7c0 tx=0x7f4b7c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ec727fa2f2136e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 0x7f4b780a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009a60 tx=0x7f4b840095c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b777fe640 1 -- 172.21.15.37:0/893460667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c09be70 con 0x55c57e32a800 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b777fe640 1 -- 172.21.15.37:0/893460667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c0056a0 con 0x55c57e32a800 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b777fe640 1 -- 172.21.15.37:0/893460667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c09ba00 con 0x55c57e32a800 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 -- 172.21.15.37:0/893460667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 msgr2=0x55c57e32cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0cd7c0 tx=0x7f4b7c021000 comp rx=0 tx=0).stop 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 -- 172.21.15.37:0/893460667 shutdown_connections 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e32cbf0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e32a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 --2- 172.21.15.37:0/893460667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e31fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 -- 172.21.15.37:0/893460667 >> 172.21.15.37:0/893460667 conn(0x55c57e31bb30 msgr2=0x55c57e31bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 -- 172.21.15.37:0/893460667 shutdown_connections 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.098+0000 7f4b961ea740 1 -- 172.21.15.37:0/893460667 wait complete. 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 -- start start 2024-04-25T21:21:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 Processor -- start 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e32a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e3273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c57e32d3d0 con 0x55c57e32a800 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.099+0000 7f4b961ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c57e21f0b0 con 0x55c57e31f620 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b961ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55c57e32d550 con 0x55c57e31ff40 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b961ea740 10 client.0 _ll_register_callbacks cb 0x55c57e20cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91278640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e32a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34020/0 (socket says 172.21.15.37:34020) 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 -- 172.21.15.37:0/3390124591 learned_addr learned my addr 172.21.15.37:0/3390124591 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e3273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 -- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 msgr2=0x55c57e3273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e3273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 -- 172.21.15.37:0/3390124591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 msgr2=0x55c57e32a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e32a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 -- 172.21.15.37:0/3390124591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e063620 con 0x55c57e32a800 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91278640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e32a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b84035000 tx=0x7f4b8402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e4289688ba0f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84054060 con 0x55c57e32a800 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f4b8404f640 con 0x55c57e32a800 2024-04-25T21:21:00.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.100+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7c0056a0 con 0x55c57e32a800 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b84034040 con 0x55c57e32a800 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 10 client.16827 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c57e3392c0 con 0x55c57e32a800 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c0056a0 con 0x55c57e32a800 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b8403a7f0 con 0x55c57e32a800 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 10 client.16827 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 20 client.16827 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e33a780 0x55c57e33cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 4 client.16827 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c57e292390 con 0x55c57e33a780 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.101+0000 7f4b91278640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e33a780 0x55c57e33cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.102+0000 7f4b91278640 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e33a780 0x55c57e33cb70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa57a0b414b41f0 server_cookie=9e352e9a723af5fc in_seq=0 out_seq=0 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.102+0000 7f4b74ff9640 10 client.16827 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:00.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.102+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3390124591 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c57e292390 con 0x55c57e33a780 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.102+0000 7f4b74ff9640 10 client.16827 handle_command_reply: tid=0 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.102+0000 7f4b961ea740 1 client.16827 shutdown 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 msgr2=0x55c57e327920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4b84035000 tx=0x7f4b8402a950 comp rx=0 tx=0).stop 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 shutdown_connections 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e33a780 0x55c57e33cb70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e32a800 0x55c57e327920 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b74ff9640 0 client.16827 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e3273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3390124591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e32a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 >> 172.21.15.37:0/3390124591 conn(0x55c57e31bb30 msgr2=0x55c57e3218b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 shutdown_connections 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 1 -- 172.21.15.37:0/3390124591 wait complete. 2024-04-25T21:21:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.103+0000 7f4b961ea740 20 client.16827 trim_cache size 0 max 16384 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 Processor -- start 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 -- start start 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e336290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e32a800 0x55c57e31ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1adb30 con 0x55c57e31ff40 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1af670 con 0x55c57e31f620 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b961ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1ae200 con 0x55c57e32a800 2024-04-25T21:21:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b90a77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34024/0 (socket says 172.21.15.37:34024) 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.118+0000 7f4b90a77640 1 -- 172.21.15.37:0/347361703 learned_addr learned my addr 172.21.15.37:0/347361703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b91278640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e336290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b91a79640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e32a800 0x55c57e31ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 -- 172.21.15.37:0/347361703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e32a800 msgr2=0x55c57e31ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e32a800 0x55c57e31ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 -- 172.21.15.37:0/347361703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 msgr2=0x55c57e336290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e336290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 -- 172.21.15.37:0/347361703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e33d1d0 con 0x55c57e31ff40 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b91278640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e336290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b90a77640 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c017000 tx=0x7f4b7c00b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21033d88336d0679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b76ffd640 1 -- 172.21.15.37:0/347361703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c09bbe0 con 0x55c57e31ff40 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b76ffd640 1 -- 172.21.15.37:0/347361703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c0056a0 con 0x55c57e31ff40 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.119+0000 7f4b76ffd640 1 -- 172.21.15.37:0/347361703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0a4d10 con 0x55c57e31ff40 2024-04-25T21:21:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- 172.21.15.37:0/347361703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 msgr2=0x55c57e3294a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c017000 tx=0x7f4b7c00b530 comp rx=0 tx=0).stop 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- 172.21.15.37:0/347361703 shutdown_connections 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e32a800 0x55c57e31ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31ff40 0x55c57e3294a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 --2- 172.21.15.37:0/347361703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e31f620 0x55c57e336290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- 172.21.15.37:0/347361703 >> 172.21.15.37:0/347361703 conn(0x55c57e31bb30 msgr2=0x55c57e31a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- 172.21.15.37:0/347361703 shutdown_connections 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- 172.21.15.37:0/347361703 wait complete. 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 -- start start 2024-04-25T21:21:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.120+0000 7f4b961ea740 1 Processor -- start 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e399600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e32a800 0x55c57e32fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1aefa0 con 0x55c57e31f620 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1ae8d0 con 0x55c57e32a800 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1abb30 con 0x55c57e31ff40 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b961ea740 10 client.0 _ll_register_callbacks cb 0x7f4b8402f3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34036/0 (socket says 172.21.15.37:34036) 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 -- 172.21.15.37:0/3775115455 learned_addr learned my addr 172.21.15.37:0/3775115455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91a79640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e32a800 0x55c57e32fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e399600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 -- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 msgr2=0x55c57e399600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e399600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 -- 172.21.15.37:0/3775115455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e32a800 msgr2=0x55c57e32fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e32a800 0x55c57e32fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 -- 172.21.15.37:0/3775115455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e330270 con 0x55c57e31f620 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e399600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b91278640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55c57e33d400 tx=0x7f4b880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbea53bcda994d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.121+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b880202e0 con 0x55c57e31f620 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8513+0+0 (secure 0 0 0) 0x7f4b88020480 con 0x55c57e31f620 2024-04-25T21:21:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7c0056a0 con 0x55c57e31f620 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b88020680 con 0x55c57e31f620 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 10 client.16833 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c57e3a0930 con 0x55c57e31f620 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c0056a0 con 0x55c57e31f620 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4b88011440 con 0x55c57e31f620 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 10 client.16833 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 20 client.16833 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e3a1030 0x55c57e3a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 4 client.16833 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c57e28c200 con 0x55c57e3a1030 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.122+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e3a1030 0x55c57e3a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.123+0000 7f4b74ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.123+0000 7f4b90a77640 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e3a1030 0x55c57e3a3450 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef8027b5cf75236 server_cookie=e0e3948af6cf8859 in_seq=0 out_seq=0 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.136+0000 7f4b74ff9640 1 -- 172.21.15.37:0/3775115455 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c57e28c200 con 0x55c57e3a1030 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.136+0000 7f4b74ff9640 10 client.16833 handle_command_reply: tid=0 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 client.16833 shutdown 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 msgr2=0x55c57e3990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x55c57e33d400 tx=0x7f4b880039c0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 shutdown_connections 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55c57e3a1030 0x55c57e3a3450 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c57e32a800 0x55c57e32fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c57e31ff40 0x55c57e399600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b74ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 --2- 172.21.15.37:0/3775115455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c57e31f620 0x55c57e3990c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 >> 172.21.15.37:0/3775115455 conn(0x55c57e31bb30 msgr2=0x55c57e31acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 shutdown_connections 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 1 -- 172.21.15.37:0/3775115455 wait complete. 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.137+0000 7f4b961ea740 20 client.16833 trim_cache size 0 max 16384 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:21:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:21:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 msgr2=0x7f4b780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 0x7f4b780a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009a60 tx=0x7f4b840095c0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 msgr2=0x7f4b8c198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d950 tx=0x7f4b7c00de10 comp rx=0 tx=0).stop 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 shutdown_connections 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4b780a3dc0 0x7f4b780a6280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b8c10c550 0x7f4b8c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b8c103aa0 0x7f4b8c198ba0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 --2- 172.21.15.37:0/3575278071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b8c103180 0x7f4b8c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 >> 172.21.15.37:0/3575278071 conn(0x7f4b8c077fd0 msgr2=0x7f4b8c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 shutdown_connections 2024-04-25T21:21:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:00.138+0000 7f4b934dc640 1 -- 172.21.15.37:0/3575278071 wait complete. 2024-04-25T21:21:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:20:59 smithi098 ceph-mon[32569]: pgmap v390: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:21:00.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:20:59 smithi037 ceph-mon[29195]: pgmap v390: 97 pgs: 97 active+clean; 5.0 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:01 smithi177 ceph-mon[33043]: pgmap v391: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:21:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:01 smithi098 ceph-mon[32569]: pgmap v391: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:21:02.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:01 smithi037 ceph-mon[29195]: pgmap v391: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:21:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:03 smithi177 ceph-mon[33043]: pgmap v392: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:21:04.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:03 smithi098 ceph-mon[32569]: pgmap v392: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:21:04.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:03 smithi037 ceph-mon[29195]: pgmap v392: 97 pgs: 97 active+clean; 5.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:21:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:05 smithi177 ceph-mon[33043]: pgmap v393: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:21:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:05 smithi098 ceph-mon[32569]: pgmap v393: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:21:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:05 smithi037 ceph-mon[29195]: pgmap v393: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:21:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:07 smithi177 ceph-mon[33043]: pgmap v394: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-25T21:21:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:07 smithi177 ceph-mon[33043]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:21:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:07 smithi098 ceph-mon[32569]: pgmap v394: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-25T21:21:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:07 smithi098 ceph-mon[32569]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:21:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:07 smithi037 ceph-mon[29195]: pgmap v394: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-25T21:21:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:07 smithi037 ceph-mon[29195]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:21:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:09 smithi177 ceph-mon[33043]: pgmap v396: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T21:21:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:09 smithi098 ceph-mon[32569]: pgmap v396: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T21:21:10.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:09 smithi037 ceph-mon[29195]: pgmap v396: 97 pgs: 97 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-25T21:21:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:11 smithi177 ceph-mon[33043]: pgmap v397: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:11 smithi098 ceph-mon[32569]: pgmap v397: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:12.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:11 smithi037 ceph-mon[29195]: pgmap v397: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- 172.21.15.37:0/4054011859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 msgr2=0x7f0bf01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/4054011859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf01014d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009900 tx=0x7f0bec02a410 comp rx=0 tx=0).stop 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- 172.21.15.37:0/4054011859 shutdown_connections 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/4054011859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/4054011859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf0101aa0 0x7f0bf0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/4054011859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf01014d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- 172.21.15.37:0/4054011859 >> 172.21.15.37:0/4054011859 conn(0x7f0bf0077fd0 msgr2=0x7f0bf00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- 172.21.15.37:0/4054011859 shutdown_connections 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- 172.21.15.37:0/4054011859 wait complete. 2024-04-25T21:21:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 Processor -- start 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.237+0000 7f0bf79e5640 1 -- start start 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf0103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf01035b0 0x7f0bf01a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01110d0 con 0x7f0bf01010f0 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf0110f50 con 0x7f0bf0106170 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf79e5640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf0111250 con 0x7f0bf01035b0 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51664/0 (socket says 172.21.15.37:51664) 2024-04-25T21:21:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 -- 172.21.15.37:0/3358866144 learned_addr learned my addr 172.21.15.37:0/3358866144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf0103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf01035b0 0x7f0bf01a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 -- 172.21.15.37:0/3358866144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf01035b0 msgr2=0x7f0bf01a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf01035b0 0x7f0bf01a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 -- 172.21.15.37:0/3358866144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 msgr2=0x7f0bf0103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf0103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 -- 172.21.15.37:0/3358866144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf01a6f30 con 0x7f0bf01010f0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.238+0000 7f0bf5781640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005940 tx=0x7f0bec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.239+0000 7f0be67fc640 1 -- 172.21.15.37:0/3358866144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec03f030 con 0x7f0bf01010f0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.239+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf01a71c0 con 0x7f0bf01010f0 2024-04-25T21:21:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.239+0000 7f0be67fc640 1 -- 172.21.15.37:0/3358866144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec003cb0 con 0x7f0bf01010f0 2024-04-25T21:21:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.239+0000 7f0be67fc640 1 -- 172.21.15.37:0/3358866144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec035070 con 0x7f0bf01010f0 2024-04-25T21:21:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.239+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf01a7700 con 0x7f0bf01010f0 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.241+0000 7f0be67fc640 1 -- 172.21.15.37:0/3358866144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0bec042030 con 0x7f0bf01010f0 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.241+0000 7f0bfa7da740 1 Processor -- start 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.241+0000 7f0bfa7da740 1 -- start start 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439620 0x555c7c439a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c446bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555c7c39e970 con 0x555c7c439f40 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c7c33b4a0 con 0x555c7c439620 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555c7c39a930 con 0x555c7c444800 2024-04-25T21:21:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0be67fc640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 0x7f0bdc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 0x7f0bdc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51670/0 (socket says 172.21.15.37:51670) 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 -- 172.21.15.37:0/108951483 learned_addr learned my addr 172.21.15.37:0/108951483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0be67fc640 1 -- 172.21.15.37:0/3358866144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8557+0+0 (secure 0 0 0) 0x7f0bec0f0db0 con 0x7f0bf01010f0 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c446bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439620 0x555c7c439a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 -- 172.21.15.37:0/108951483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 msgr2=0x555c7c446bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c446bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 -- 172.21.15.37:0/108951483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439620 msgr2=0x555c7c439a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439620 0x555c7c439a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 -- 172.21.15.37:0/108951483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7c0b1210 con 0x555c7c439f40 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5781640 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec0b21c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33d65ff3af3ff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.242+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 0x7f0bdc0a62b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0be8007fd0 tx=0x7f0be8007c60 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bdb7fe640 1 -- 172.21.15.37:0/108951483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec049050 con 0x555c7c439f40 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bdb7fe640 1 -- 172.21.15.37:0/108951483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec0bd430 con 0x555c7c439f40 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bdb7fe640 1 -- 172.21.15.37:0/108951483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0b93a0 con 0x555c7c439f40 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 -- 172.21.15.37:0/108951483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 msgr2=0x555c7c4442c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0bec006010 tx=0x7f0bec0b21c0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 -- 172.21.15.37:0/108951483 shutdown_connections 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c446bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c4442c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/108951483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439620 0x555c7c439a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 -- 172.21.15.37:0/108951483 >> 172.21.15.37:0/108951483 conn(0x555c7c435b30 msgr2=0x555c7c435f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 -- 172.21.15.37:0/108951483 shutdown_connections 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.243+0000 7f0bfa7da740 1 -- 172.21.15.37:0/108951483 wait complete. 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 -- start start 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 Processor -- start 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439f40 0x555c7c441480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c441a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555c7c4471f0 con 0x555c7c439620 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555c7c3390b0 con 0x555c7c439f40 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555c7c447370 con 0x555c7c444800 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.244+0000 7f0bfa7da740 10 client.0 _ll_register_callbacks cb 0x555c7c326d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51674/0 (socket says 172.21.15.37:51674) 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 -- 172.21.15.37:0/2313058250 learned_addr learned my addr 172.21.15.37:0/2313058250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c441a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439f40 0x555c7c441480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 -- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 msgr2=0x555c7c441a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c441a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 -- 172.21.15.37:0/2313058250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439f40 msgr2=0x555c7c441480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439f40 0x555c7c441480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 -- 172.21.15.37:0/2313058250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7c17d620 con 0x555c7c439620 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c441a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bf5781640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0eeae0 tx=0x7f0bec0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a750d155b87dfc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec049050 con 0x555c7c439620 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8557+0+0 (secure 0 0 0) 0x7f0bec0c38d0 con 0x555c7c439620 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec0bd430 con 0x555c7c439620 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bec0bb110 con 0x555c7c439620 2024-04-25T21:21:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bfa7da740 10 client.16845 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c7c4471f0 con 0x555c7c439620 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.245+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bec0bd430 con 0x555c7c439620 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bec03bbf0 con 0x555c7c439620 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 10 client.16845 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 20 client.16845 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 4 client.16845 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c7c3ac390 con 0x555c7c454390 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.246+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.247+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4567a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b022aeb0abf9b3 server_cookie=65971287ac28621e in_seq=0 out_seq=0 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.247+0000 7f0bd8ff9640 10 client.16845 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.247+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/2313058250 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c7c3ac390 con 0x555c7c454390 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.247+0000 7f0bd8ff9640 10 client.16845 handle_command_reply: tid=0 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.247+0000 7f0bfa7da740 1 client.16845 shutdown 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 msgr2=0x555c7c444190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0eeae0 tx=0x7f0bec0045a0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 shutdown_connections 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4567a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c444800 0x555c7c441a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bd8ff9640 0 client.16845 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c439f40 0x555c7c441480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2313058250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c444190 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 >> 172.21.15.37:0/2313058250 conn(0x555c7c435b30 msgr2=0x555c7c43b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 shutdown_connections 2024-04-25T21:21:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2313058250 wait complete. 2024-04-25T21:21:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.248+0000 7f0bfa7da740 20 client.16845 trim_cache size 0 max 16384 2024-04-25T21:21:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 Processor -- start 2024-04-25T21:21:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 -- start start 2024-04-25T21:21:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439620 0x555c7c450000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c45c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01afcb0 con 0x555c7c439f40 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01b17f0 con 0x555c7c444800 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01b0380 con 0x555c7c439620 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bf4f80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.263+0000 7f0bf4f80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51680/0 (socket says 172.21.15.37:51680) 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 -- 172.21.15.37:0/2636304528 learned_addr learned my addr 172.21.15.37:0/2636304528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c45c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf5781640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439620 0x555c7c450000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 -- 172.21.15.37:0/2636304528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439620 msgr2=0x555c7c450000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439620 0x555c7c450000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 -- 172.21.15.37:0/2636304528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 msgr2=0x555c7c45c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c45c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 -- 172.21.15.37:0/2636304528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7c440430 con 0x555c7c439f40 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c45c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0bf01045c0 tx=0x7f0be000b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e54b5db3d8c5520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bdaffd640 1 -- 172.21.15.37:0/2636304528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0014b20 con 0x555c7c439f40 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bdaffd640 1 -- 172.21.15.37:0/2636304528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be00054d0 con 0x555c7c439f40 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.264+0000 7f0bdaffd640 1 -- 172.21.15.37:0/2636304528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be001b860 con 0x555c7c439f40 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2636304528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 msgr2=0x555c7c434b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0bf01045c0 tx=0x7f0be000b5a0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2636304528 shutdown_connections 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c45c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439f40 0x555c7c434b30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/2636304528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439620 0x555c7c450000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2636304528 >> 172.21.15.37:0/2636304528 conn(0x555c7c435b30 msgr2=0x555c7c434420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2636304528 shutdown_connections 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- 172.21.15.37:0/2636304528 wait complete. 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 -- start start 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.265+0000 7f0bfa7da740 1 Processor -- start 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439f40 0x555c7c44a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c44a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01b1120 con 0x555c7c439620 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01b0a50 con 0x555c7c444800 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf01adcb0 con 0x555c7c439f40 2024-04-25T21:21:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 10 client.0 _ll_register_callbacks cb 0x7f0bec0b9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51686/0 (socket says 172.21.15.37:51686) 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 -- 172.21.15.37:0/790798321 learned_addr learned my addr 172.21.15.37:0/790798321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439f40 0x555c7c44a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c44a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 -- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439f40 msgr2=0x555c7c44a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439f40 0x555c7c44a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 -- 172.21.15.37:0/790798321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 msgr2=0x555c7c44a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c44a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 -- 172.21.15.37:0/790798321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c7c441d70 con 0x555c7c439620 2024-04-25T21:21:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5f82640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c44a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bf5781640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0bafe0 tx=0x7f0bec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0256875ef480a84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec049070 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8557+0+0 (secure 0 0 0) 0x7f0bec03bbf0 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.266+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be00054d0 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bec0d62b0 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 10 client.16851 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555c7c439d70 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be00054d0 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0bec0c5510 con 0x555c7c439620 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 10 client.16851 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 20 client.16851 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 4 client.16851 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c7c3a6200 con 0x555c7c454390 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.267+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.268+0000 7f0bd8ff9640 10 client.16851 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.268+0000 7f0bf4f80640 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4bca20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2586ec9ce452f1d server_cookie=2aec840c6ff14071 in_seq=0 out_seq=0 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.282+0000 7f0bd8ff9640 1 -- 172.21.15.37:0/790798321 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c7c3a6200 con 0x555c7c454390 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.282+0000 7f0bd8ff9640 10 client.16851 handle_command_reply: tid=0 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 client.16851 shutdown 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 msgr2=0x555c7c449e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0bafe0 tx=0x7f0bec02a410 comp rx=0 tx=0).stop 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 shutdown_connections 2024-04-25T21:21:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555c7c454390 0x555c7c4bca20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555c7c444800 0x555c7c44a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555c7c439f40 0x555c7c44a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bd8ff9640 0 client.16851 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 --2- 172.21.15.37:0/790798321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555c7c439620 0x555c7c449e80 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 >> 172.21.15.37:0/790798321 conn(0x555c7c435b30 msgr2=0x555c7c434cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 shutdown_connections 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 1 -- 172.21.15.37:0/790798321 wait complete. 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.283+0000 7f0bfa7da740 20 client.16851 trim_cache size 0 max 16384 2024-04-25T21:21:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:21:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:21:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:21:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 msgr2=0x7f0bdc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 0x7f0bdc0a62b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0be8007fd0 tx=0x7f0be8007c60 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 msgr2=0x7f0bf0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005940 tx=0x7f0bec0042f0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 shutdown_connections 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0bdc0a3df0 0x7f0bdc0a62b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bf01035b0 0x7f0bf01a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0bf0106170 0x7f0bf0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 --2- 172.21.15.37:0/3358866144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0bf01010f0 0x7f0bf0102b30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 >> 172.21.15.37:0/3358866144 conn(0x7f0bf0077fd0 msgr2=0x7f0bf00698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 shutdown_connections 2024-04-25T21:21:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:13.284+0000 7f0bf79e5640 1 -- 172.21.15.37:0/3358866144 wait complete. 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:375/600: open fd = 3 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:376/600: open fd = 3 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:377/600: open fd = 3 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.651 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:378/600: open fd = 3 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:379/600: open fd = 3 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:380/600: open fd = 3 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.652 INFO:tasks.workunit.client.0.smithi037.stdout:381/600: open fd = 3 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:382/600: open fd = 3 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:383/600: open fd = 3 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:384/600: open fd = 3 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:385/600: open fd = 3 2024-04-25T21:21:13.653 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:386/600: open fd = 3 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:387/600: open fd = 3 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:388/600: open fd = 3 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.654 INFO:tasks.workunit.client.0.smithi037.stdout:389/600: open fd = 3 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:390/600: open fd = 3 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:391/600: open fd = 3 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:392/600: open fd = 3 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.655 INFO:tasks.workunit.client.0.smithi037.stdout:393/600: open fd = 3 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:394/600: open fd = 3 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:395/600: open fd = 3 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:396/600: open fd = 3 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.656 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:397/600: open fd = 3 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:398/600: open fd = 3 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:399/600: open fd = 3 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.657 INFO:tasks.workunit.client.0.smithi037.stdout:400/600: open fd = 3 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:401/600: open fd = 3 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:402/600: open fd = 3 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.658 INFO:tasks.workunit.client.0.smithi037.stdout:403/600: open fd = 3 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:404/600: open fd = 3 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:405/600: open fd = 3 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:406/600: open fd = 3 2024-04-25T21:21:13.659 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:407/600: open fd = 3 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:408/600: open fd = 3 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:409/600: open fd = 3 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.660 INFO:tasks.workunit.client.0.smithi037.stdout:410/600: open fd = 3 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:411/600: open fd = 3 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:412/600: open fd = 3 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:413/600: open fd = 3 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.661 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:414/600: open fd = 3 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:415/600: open fd = 3 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:416/600: open fd = 3 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:417/600: open fd = 3 2024-04-25T21:21:13.662 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:418/600: open fd = 3 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:419/600: open fd = 3 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.663 INFO:tasks.workunit.client.0.smithi037.stdout:420/600: open fd = 3 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:421/600: open fd = 3 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:422/600: open fd = 3 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:423/600: open fd = 3 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.664 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:424/600: open fd = 3 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:425/600: open fd = 3 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:426/600: open fd = 3 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:427/600: open fd = 3 2024-04-25T21:21:13.665 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:428/600: open fd = 3 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:429/600: open fd = 3 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:430/600: open fd = 3 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.666 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:431/600: open fd = 3 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:432/600: open fd = 3 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:433/600: open fd = 3 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:434/600: open fd = 3 2024-04-25T21:21:13.667 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:435/600: open fd = 3 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:436/600: open fd = 3 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:21:13.668 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:21:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:13 smithi177 ceph-mon[33043]: pgmap v398: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:13 smithi098 ceph-mon[32569]: pgmap v398: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:13 smithi037 ceph-mon[29195]: pgmap v398: 97 pgs: 97 active+clean; 5.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-25T21:21:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:15 smithi177 ceph-mon[33043]: pgmap v399: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:15 smithi177 ceph-mon[33043]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:21:16.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:15 smithi098 ceph-mon[32569]: pgmap v399: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:16.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:15 smithi098 ceph-mon[32569]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:21:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:15 smithi037 ceph-mon[29195]: pgmap v399: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:15 smithi037 ceph-mon[29195]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:21:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:17 smithi177 ceph-mon[33043]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-04-25T21:21:18.131 INFO:tasks.quiescer.fs.[cephfs]:Total duration (53.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:21:18.132 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.55 2024-04-25T21:21:18.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:21:18.160 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:17 smithi037 ceph-mon[29195]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-04-25T21:21:18.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:17 smithi098 ceph-mon[32569]: pgmap v401: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-04-25T21:21:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:20.083 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:19 smithi037 ceph-mon[29195]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:19 smithi177 ceph-mon[33043]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:20.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:19 smithi098 ceph-mon[32569]: pgmap v402: 97 pgs: 97 active+clean; 1.9 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-25T21:21:20.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 -- 172.21.15.37:0/2727100307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f60101d20 msgr2=0x7f3f601021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:20.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/2727100307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f60101d20 0x7f3f601021a0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009990 tx=0x7f3f54009ca0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 -- 172.21.15.37:0/2727100307 shutdown_connections 2024-04-25T21:21:20.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/2727100307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f601026e0 0x7f3f6010e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/2727100307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60106990 0x7f3f60106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/2727100307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f60101d20 0x7f3f601021a0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 -- 172.21.15.37:0/2727100307 >> 172.21.15.37:0/2727100307 conn(0x7f3f600788f0 msgr2=0x7f3f60078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 -- 172.21.15.37:0/2727100307 shutdown_connections 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.322+0000 7f3f67ef7700 1 -- 172.21.15.37:0/2727100307 wait complete. 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 Processor -- start 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 -- start start 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60101d20 0x7f3f60199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f60106990 0x7f3f6019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60111ae0 con 0x7f3f601026e0 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60111960 con 0x7f3f60106990 2024-04-25T21:21:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f67ef7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60111c60 con 0x7f3f60101d20 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f65492700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f65492700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58706/0 (socket says 172.21.15.37:58706) 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.323+0000 7f3f65492700 1 -- 172.21.15.37:0/1375965990 learned_addr learned my addr 172.21.15.37:0/1375965990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65c93700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60101d20 0x7f3f60199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f66494700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f60106990 0x7f3f6019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 -- 172.21.15.37:0/1375965990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60101d20 msgr2=0x7f3f60199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60101d20 0x7f3f60199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 -- 172.21.15.37:0/1375965990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f60106990 msgr2=0x7f3f6019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f60106990 0x7f3f6019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 -- 172.21.15.37:0/1375965990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6019e040 con 0x7f3f601026e0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f65492700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54027070 con 0x7f3f601026e0 2024-04-25T21:21:20.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540040e0 con 0x7f3f601026e0 2024-04-25T21:21:20.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6019e2d0 con 0x7f3f601026e0 2024-04-25T21:21:20.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400f460 con 0x7f3f601026e0 2024-04-25T21:21:20.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.324+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601a5bb0 con 0x7f3f601026e0 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.325+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f60103080 con 0x7f3f601026e0 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.326+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f5402a070 con 0x7f3f601026e0 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.327+0000 7f3f52ffd700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 0x7f3f4c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.327+0000 7f3f65c93700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 0x7f3f4c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.327+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f3f540d1bb0 con 0x7f3f601026e0 2024-04-25T21:21:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.328+0000 7f3f65c93700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 0x7f3f4c0a62f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005950 tx=0x7f3f5c00a300 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:20.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.331+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f54031070 con 0x7f3f601026e0 2024-04-25T21:21:20.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.589+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f6019a390 con 0x7f3f601026e0 2024-04-25T21:21:20.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.590+0000 7f3f52ffd700 1 -- 172.21.15.37:0/1375965990 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f3f54013070 con 0x7f3f601026e0 2024-04-25T21:21:20.590 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:21:20.590 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:21:20.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 msgr2=0x7f3f4c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 0x7f3f4c0a62f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c005950 tx=0x7f3f5c00a300 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 msgr2=0x7f3f60199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3f54000c00 tx=0x7f3f540047b0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 shutdown_connections 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f4c0a3e30 0x7f3f4c0a62f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f60106990 0x7f3f6019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f60101d20 0x7f3f60199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 --2- 172.21.15.37:0/1375965990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f601026e0 0x7f3f60199570 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 >> 172.21.15.37:0/1375965990 conn(0x7f3f600788f0 msgr2=0x7f3f6006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 shutdown_connections 2024-04-25T21:21:20.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:20.593+0000 7f3f67ef7700 1 -- 172.21.15.37:0/1375965990 wait complete. 2024-04-25T21:21:20.594 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:21:21.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:20 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1375965990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:21.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:20 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1375965990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:21.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:20 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1375965990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:21.293 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 86.55 --await-for 42.0' 2024-04-25T21:21:21.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 86.55 --await-for 42.0 2024-04-25T21:21:21.587 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:21 smithi037 ceph-mon[29195]: pgmap v403: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:21 smithi177 ceph-mon[33043]: pgmap v403: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:21 smithi098 ceph-mon[32569]: pgmap v403: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 -- 172.21.15.37:0/30004268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c102af0 msgr2=0x7f231c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 --2- 172.21.15.37:0/30004268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c102af0 0x7f231c102f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f23100099c0 tx=0x7f2310009cd0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 -- 172.21.15.37:0/30004268 shutdown_connections 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 --2- 172.21.15.37:0/30004268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 0x7f231c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 --2- 172.21.15.37:0/30004268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c108a30 0x7f231c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 --2- 172.21.15.37:0/30004268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c102af0 0x7f231c102f70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 -- 172.21.15.37:0/30004268 >> 172.21.15.37:0/30004268 conn(0x7f231c079b10 msgr2=0x7f231c079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 -- 172.21.15.37:0/30004268 shutdown_connections 2024-04-25T21:21:23.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 -- 172.21.15.37:0/30004268 wait complete. 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.342+0000 7f2323466700 1 Processor -- start 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 -- start start 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c102af0 0x7f231c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 0x7f231c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112c00 con 0x7f231c108a30 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112a80 con 0x7f231c1034b0 2024-04-25T21:21:23.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2323466700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f231c112d80 con 0x7f231c102af0 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58726/0 (socket says 172.21.15.37:58726) 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 -- 172.21.15.37:0/2219022565 learned_addr learned my addr 172.21.15.37:0/2219022565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321202700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c102af0 0x7f231c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2320a01700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 0x7f231c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 -- 172.21.15.37:0/2219022565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c102af0 msgr2=0x7f231c199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c102af0 0x7f231c199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 -- 172.21.15.37:0/2219022565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 msgr2=0x7f231c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 0x7f231c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 -- 172.21.15.37:0/2219022565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c19e040 con 0x7f231c108a30 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.343+0000 7f2321a03700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f231400e940 tx=0x7f231400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.344+0000 7f231a7fc700 1 -- 172.21.15.37:0/2219022565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314004a30 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.344+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c19e330 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.344+0000 7f231a7fc700 1 -- 172.21.15.37:0/2219022565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23140054d0 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.344+0000 7f231a7fc700 1 -- 172.21.15.37:0/2219022565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231400ccf0 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.344+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1a5bb0 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.346+0000 7f231a7fc700 1 -- 172.21.15.37:0/2219022565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2314005020 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.346+0000 7f231a7fc700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 0x7f23040a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.347+0000 7f2321202700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 0x7f23040a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.347+0000 7f231a7fc700 1 -- 172.21.15.37:0/2219022565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f23140d09f0 con 0x7f231c108a30 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.347+0000 7f2321202700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 0x7f23040a6310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f230c009550 tx=0x7f230c0091b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.347+0000 7f232dcefb80 1 Processor -- start 2024-04-25T21:21:23.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.347+0000 7f232dcefb80 1 -- start start 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d940461ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d94046c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d9403b4e20 con 0x55d94046cf00 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d94035bac0 con 0x55d9404616e0 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f232dcefb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d9403b0de0 con 0x55d940462000 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58740/0 (socket says 172.21.15.37:58740) 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 -- 172.21.15.37:0/4064095758 learned_addr learned my addr 172.21.15.37:0/4064095758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2320a01700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d94046c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321202700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d940461ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 -- 172.21.15.37:0/4064095758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 msgr2=0x55d94046c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d94046c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 -- 172.21.15.37:0/4064095758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 msgr2=0x55d940461ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d940461ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 -- 172.21.15.37:0/4064095758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94031f8d0 con 0x55d94046cf00 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321202700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d940461ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f2321a03700 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f23140a5070 tx=0x7f2314098d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63224624b1cea1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f23037fe700 1 -- 172.21.15.37:0/4064095758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231409c930 con 0x55d94046cf00 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f23037fe700 1 -- 172.21.15.37:0/4064095758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23140a0ce0 con 0x55d94046cf00 2024-04-25T21:21:23.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.348+0000 7f23037fe700 1 -- 172.21.15.37:0/4064095758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314098fa0 con 0x55d94046cf00 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 -- 172.21.15.37:0/4064095758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 msgr2=0x55d94046f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f23140a5070 tx=0x7f2314098d00 comp rx=0 tx=0).stop 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 -- 172.21.15.37:0/4064095758 shutdown_connections 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d94046f2f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d940461ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 --2- 172.21.15.37:0/4064095758 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d94046c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 -- 172.21.15.37:0/4064095758 >> 172.21.15.37:0/4064095758 conn(0x55d94038ea20 msgr2=0x55d94045dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 -- 172.21.15.37:0/4064095758 shutdown_connections 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.349+0000 7f232dcefb80 1 -- 172.21.15.37:0/4064095758 wait complete. 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 -- start start 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 Processor -- start 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d94046c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d940469a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d94046f990 con 0x55d94046cf00 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55d94036e740 con 0x55d9404616e0 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55d94046fb10 con 0x55d940462000 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f232dcefb80 10 client.0 _ll_register_callbacks cb 0x55d94034dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f2321a03700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58754/0 (socket says 172.21.15.37:58754) 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f2320a01700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d940469a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.350+0000 7f2321a03700 1 -- 172.21.15.37:0/94274599 learned_addr learned my addr 172.21.15.37:0/94274599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321202700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d94046c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 -- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 msgr2=0x55d940469a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d940469a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 -- 172.21.15.37:0/94274599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 msgr2=0x55d94046c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d94046c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 -- 172.21.15.37:0/94274599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d940302a60 con 0x55d94046cf00 2024-04-25T21:21:23.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321202700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d94046c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2321a03700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23140ce660 tx=0x7f2314005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90c9a2c108c736f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2300ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231409c930 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f231409ac30 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23140a0ce0 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2314067220 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f232dcefb80 10 client.16869 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.351+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d94047a280 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23140a0ce0 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f23140a5110 con 0x55d94046cf00 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 10 client.16869 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 10 client.16869 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 20 client.16869 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d94047eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 4 client.16869 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9403c2610 con 0x55d94047c750 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.352+0000 7f2321202700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d94047eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.353+0000 7f2300ff9700 10 client.16869 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.353+0000 7f2321202700 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d94047eb40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a46bdd90b99aacd server_cookie=91f48791d2114776 in_seq=0 out_seq=0 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.353+0000 7f2300ff9700 1 -- 172.21.15.37:0/94274599 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9403c2610 con 0x55d94047c750 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.353+0000 7f2300ff9700 10 client.16869 handle_command_reply: tid=0 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 client.16869 shutdown 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 msgr2=0x55d940469f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23140ce660 tx=0x7f2314005b40 comp rx=0 tx=0).stop 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 shutdown_connections 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d94047eb40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f2300ff9700 0 client.16869 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:23.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d94046cf00 0x55d940469f70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d9404616e0 0x55d94046c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 --2- 172.21.15.37:0/94274599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d940469a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 >> 172.21.15.37:0/94274599 conn(0x55d94038ea20 msgr2=0x55d940464080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 shutdown_connections 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 1 -- 172.21.15.37:0/94274599 wait complete. 2024-04-25T21:21:23.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.354+0000 7f232dcefb80 20 client.16869 trim_cache size 0 max 16384 2024-04-25T21:21:23.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 Processor -- start 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 -- start start 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 0x55d94046c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d94046cf00 0x55d940481b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1ad250 con 0x55d9404616e0 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1aed90 con 0x55d940462000 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f232dcefb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1ad920 con 0x55d94046cf00 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58760/0 (socket says 172.21.15.37:58760) 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 -- 172.21.15.37:0/2616860503 learned_addr learned my addr 172.21.15.37:0/2616860503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321a03700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d94046cf00 0x55d940481b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2320a01700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 0x55d94046c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 -- 172.21.15.37:0/2616860503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d94046cf00 msgr2=0x55d940481b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d94046cf00 0x55d940481b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 -- 172.21.15.37:0/2616860503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 msgr2=0x55d94046c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 0x55d94046c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2321202700 1 -- 172.21.15.37:0/2616860503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d940467fc0 con 0x55d9404616e0 2024-04-25T21:21:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.374+0000 7f2320a01700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 0x55d94046c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.375+0000 7f2321202700 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f230c0047e0 tx=0x7f230c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5ef3a93e2340d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.375+0000 7f2302ffd700 1 -- 172.21.15.37:0/2616860503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c007c70 con 0x55d9404616e0 2024-04-25T21:21:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.375+0000 7f2302ffd700 1 -- 172.21.15.37:0/2616860503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c007e10 con 0x55d9404616e0 2024-04-25T21:21:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.375+0000 7f2302ffd700 1 -- 172.21.15.37:0/2616860503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c022680 con 0x55d9404616e0 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- 172.21.15.37:0/2616860503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 msgr2=0x55d940468d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f230c0047e0 tx=0x7f230c013000 comp rx=0 tx=0).stop 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- 172.21.15.37:0/2616860503 shutdown_connections 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d94046cf00 0x55d940481b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d940468d20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2616860503 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d940462000 0x55d94046c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- 172.21.15.37:0/2616860503 >> 172.21.15.37:0/2616860503 conn(0x55d94038ea20 msgr2=0x55d940478630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- 172.21.15.37:0/2616860503 shutdown_connections 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- 172.21.15.37:0/2616860503 wait complete. 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 -- start start 2024-04-25T21:21:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.376+0000 7f232dcefb80 1 Processor -- start 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d9404dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 0x55d9404d68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1ae6c0 con 0x55d9404616e0 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1adff0 con 0x55d94046cf00 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1ab210 con 0x55d940462000 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f232dcefb80 10 client.0 _ll_register_callbacks cb 0x55d94034dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:23.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2320a01700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d9404dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321202700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321a03700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 0x55d9404d68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2320a01700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d9404dc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:43916/0 (socket says 172.21.15.37:43916) 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2320a01700 1 -- 172.21.15.37:0/2522631622 learned_addr learned my addr 172.21.15.37:0/2522631622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321202700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58776/0 (socket says 172.21.15.37:58776) 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321a03700 1 -- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 msgr2=0x55d9404dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321a03700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d9404dc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.377+0000 7f2321a03700 1 -- 172.21.15.37:0/2522631622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 msgr2=0x55d9404dc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:23.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2321a03700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2321a03700 1 -- 172.21.15.37:0/2522631622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9404d6fc0 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2321202700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2321a03700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 0x55d9404d68c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23140a5760 tx=0x7f231400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b0c9934441cfad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2300ff9700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231409c290 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230c007e10 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f231409cb90 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2314098bd0 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f232dcefb80 10 client.26153 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9404e7ee0 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.378+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c007e10 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f231409bd80 con 0x55d94046cf00 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 10 client.26153 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 10 client.26153 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:21:23.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 20 client.26153 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d9404ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 4 client.26153 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.55, "await_for": 42.0}] 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.55, "await_for": 42.0}) -- 0x55d9403bc6b0 con 0x55d94047c750 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f2321202700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d9404ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f2300ff9700 10 client.26153 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:23.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.379+0000 7f2321202700 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d9404ea4d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2aa93dacf2241e server_cookie=6d86b619182126d1 in_seq=0 out_seq=0 2024-04-25T21:21:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f2300ff9700 1 -- 172.21.15.37:0/2522631622 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d9403bc6b0 con 0x55d94047c750 2024-04-25T21:21:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f2300ff9700 10 client.26153 handle_command_reply: tid=0 2024-04-25T21:21:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 client.26153 shutdown 2024-04-25T21:21:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 msgr2=0x55d9404d68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 0x55d9404d68c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f23140a5760 tx=0x7f231400c450 comp rx=0 tx=0).stop 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 shutdown_connections 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55d94047c750 0x55d9404ea4d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55d94046cf00 0x55d9404d68c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d9404616e0 0x55d9404dc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f2300ff9700 0 client.26153 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 --2- 172.21.15.37:0/2522631622 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d940462000 0x55d9404dc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 >> 172.21.15.37:0/2522631622 conn(0x55d94038ea20 msgr2=0x55d94046b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 shutdown_connections 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 1 -- 172.21.15.37:0/2522631622 wait complete. 2024-04-25T21:21:23.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.394+0000 7f232dcefb80 20 client.26153 trim_cache size 0 max 16384 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 34, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "efdf936b": { 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 34, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.5, 2024-04-25T21:21:23.397 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:23.398 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 msgr2=0x7f23040a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 0x7f23040a6310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f230c009550 tx=0x7f230c0091b0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 msgr2=0x7f231c19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f231400e940 tx=0x7f231400ec50 comp rx=0 tx=0).stop 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 shutdown_connections 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f23040a3e50 0x7f23040a6310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f231c108a30 0x7f231c19d900 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f231c102af0 0x7f231c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 --2- 172.21.15.37:0/2219022565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f231c1034b0 0x7f231c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 >> 172.21.15.37:0/2219022565 conn(0x7f231c079b10 msgr2=0x7f231c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 shutdown_connections 2024-04-25T21:21:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:23.395+0000 7f2323466700 1 -- 172.21.15.37:0/2219022565 wait complete. 2024-04-25T21:21:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:23 smithi177 ceph-mon[33043]: pgmap v404: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:24.089 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:23 smithi037 ceph-mon[29195]: pgmap v404: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:24.120 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'efdf936b', quiesce took 6.0 seconds. Will release after: 26.55 2024-04-25T21:21:24.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:23 smithi098 ceph-mon[32569]: pgmap v404: 97 pgs: 97 active+clean; 2.1 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:21:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:25 smithi177 ceph-mon[33043]: pgmap v405: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-25T21:21:26.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:25 smithi098 ceph-mon[32569]: pgmap v405: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-25T21:21:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:25 smithi037 ceph-mon[29195]: pgmap v405: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-25T21:21:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.385+0000 7f6ec742a640 1 -- 172.21.15.37:0/996974050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec01031e0 msgr2=0x7f6ec01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.385+0000 7f6ec742a640 1 --2- 172.21.15.37:0/996974050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec01031e0 0x7f6ec01035c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009900 tx=0x7f6ebc02a510 comp rx=0 tx=0).stop 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.385+0000 7f6ec742a640 1 -- 172.21.15.37:0/996974050 shutdown_connections 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.385+0000 7f6ec742a640 1 --2- 172.21.15.37:0/996974050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec010c520 0x7f6ec010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.385+0000 7f6ec742a640 1 --2- 172.21.15.37:0/996974050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 --2- 172.21.15.37:0/996974050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec01031e0 0x7f6ec01035c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 -- 172.21.15.37:0/996974050 >> 172.21.15.37:0/996974050 conn(0x7f6ec0077fd0 msgr2=0x7f6ec00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 -- 172.21.15.37:0/996974050 shutdown_connections 2024-04-25T21:21:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 -- 172.21.15.37:0/996974050 wait complete. 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 Processor -- start 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 -- start start 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.386+0000 7f6ec742a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec01031e0 0x7f6ec0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01121b0 con 0x7f6ec010c520 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0112030 con 0x7f6ec01031e0 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec0112330 con 0x7f6ec0103b00 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58794/0 (socket says 172.21.15.37:58794) 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 -- 172.21.15.37:0/3729546095 learned_addr learned my addr 172.21.15.37:0/3729546095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec01031e0 0x7f6ec0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 -- 172.21.15.37:0/3729546095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 msgr2=0x7f6ec0198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec0198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 -- 172.21.15.37:0/3729546095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec01031e0 msgr2=0x7f6ec0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec01031e0 0x7f6ec0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 -- 172.21.15.37:0/3729546095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec019d6f0 con 0x7f6ec010c520 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800eaa0 tx=0x7f6eb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6eb67fc640 1 -- 172.21.15.37:0/3729546095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb800cdb0 con 0x7f6ec010c520 2024-04-25T21:21:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6eb67fc640 1 -- 172.21.15.37:0/3729546095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb8005020 con 0x7f6ec010c520 2024-04-25T21:21:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec019d9e0 con 0x7f6ec010c520 2024-04-25T21:21:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6eb67fc640 1 -- 172.21.15.37:0/3729546095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80116d0 con 0x7f6ec010c520 2024-04-25T21:21:26.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.387+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec01a5270 con 0x7f6ec010c520 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.390+0000 7f6eb67fc640 1 -- 172.21.15.37:0/3729546095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6eb80051d0 con 0x7f6ec010c520 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.390+0000 7f6eca21d740 1 Processor -- start 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 -- start start 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0f40 0x560a32efb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32efdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a32e55970 con 0x560a32ef0620 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a32df24a0 con 0x560a32ef0f40 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eca21d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a32e51930 con 0x560a32efb800 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58798/0 (socket says 172.21.15.37:58798) 2024-04-25T21:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 -- 172.21.15.37:0/1729147339 learned_addr learned my addr 172.21.15.37:0/1729147339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0f40 0x560a32efb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 -- 172.21.15.37:0/1729147339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 msgr2=0x560a32efdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32efdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 -- 172.21.15.37:0/1729147339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0f40 msgr2=0x560a32efb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0f40 0x560a32efb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 -- 172.21.15.37:0/1729147339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a32b68210 con 0x560a32ef0620 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6eb67fc640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 0x7f6eac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.391+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00058c0 tx=0x7f6eb0011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a007b73a35222fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.392+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 0x7f6eac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.392+0000 7f6eb67fc640 1 -- 172.21.15.37:0/3729546095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f6eb80d0960 con 0x7f6ec010c520 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.392+0000 7f6eab7fe640 1 -- 172.21.15.37:0/1729147339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011e60 con 0x560a32ef0620 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.392+0000 7f6eab7fe640 1 -- 172.21.15.37:0/1729147339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00128a0 con 0x560a32ef0620 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.392+0000 7f6eab7fe640 1 -- 172.21.15.37:0/1729147339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000f880 con 0x560a32ef0620 2024-04-25T21:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- 172.21.15.37:0/1729147339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 msgr2=0x560a32ef0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 0x7f6eac0a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc031440 tx=0x7f6ebc005c00 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00058c0 tx=0x7f6eb0011750 comp rx=0 tx=0).stop 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- 172.21.15.37:0/1729147339 shutdown_connections 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32efdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0f40 0x560a32efb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 --2- 172.21.15.37:0/1729147339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0620 0x560a32ef0a00 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- 172.21.15.37:0/1729147339 >> 172.21.15.37:0/1729147339 conn(0x560a32eecb30 msgr2=0x560a32eecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- 172.21.15.37:0/1729147339 shutdown_connections 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- 172.21.15.37:0/1729147339 wait complete. 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 -- start start 2024-04-25T21:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.393+0000 7f6eca21d740 1 Processor -- start 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0620 0x560a32efb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32efb800 0x560a32ef89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560a32efe430 con 0x560a32ef0f40 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560a32df00b0 con 0x560a32efb800 2024-04-25T21:21:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560a32efe5b0 con 0x560a32ef0620 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6eca21d740 10 client.0 _ll_register_callbacks cb 0x560a32dddd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58802/0 (socket says 172.21.15.37:58802) 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 -- 172.21.15.37:0/3070370629 learned_addr learned my addr 172.21.15.37:0/3070370629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32efb800 0x560a32ef89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0620 0x560a32efb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 -- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0620 msgr2=0x560a32efb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0620 0x560a32efb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 -- 172.21.15.37:0/3070370629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32efb800 msgr2=0x560a32ef89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32efb800 0x560a32ef89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 -- 172.21.15.37:0/3070370629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a32c34620 con 0x560a32ef0f40 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000f400 tx=0x7f6eb00126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f3f1a2152d915a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.394+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002a060 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f6eb0012c60 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb00128a0 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6eb0011e60 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 10 client.16887 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a32f0a360 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00128a0 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb0004930 con 0x560a32ef0f40 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 10 client.16887 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 20 client.16887 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f0b7c0 0x560a32f0dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 4 client.16887 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.395+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a32e63390 con 0x560a32f0b7c0 2024-04-25T21:21:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.396+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f0b7c0 0x560a32f0dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.396+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f0b7c0 0x560a32f0dbb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a19f3ccf3430ebe server_cookie=a9f58a5ec6cb1657 in_seq=0 out_seq=0 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.396+0000 7f6ea8ff9640 10 client.16887 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/3070370629 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a32e63390 con 0x560a32f0b7c0 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6ea8ff9640 10 client.16887 handle_command_reply: tid=0 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 client.16887 shutdown 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 msgr2=0x560a32ef8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000f400 tx=0x7f6eb00126a0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 shutdown_connections 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f0b7c0 0x560a32f0dbb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32efb800 0x560a32ef89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6ea8ff9640 0 client.16887 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:26.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32ef8450 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 --2- 172.21.15.37:0/3070370629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0620 0x560a32efb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 >> 172.21.15.37:0/3070370629 conn(0x560a32eecb30 msgr2=0x560a32ef28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 shutdown_connections 2024-04-25T21:21:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 1 -- 172.21.15.37:0/3070370629 wait complete. 2024-04-25T21:21:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.397+0000 7f6eca21d740 20 client.16887 trim_cache size 0 max 16384 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 Processor -- start 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 -- start start 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32efac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 0x560a32f10be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01adc10 con 0x560a32efb800 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01af750 con 0x560a32ef0620 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6eca21d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01ae2e0 con 0x560a32ef0f40 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.412+0000 7f6ec59c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58816/0 (socket says 172.21.15.37:58816) 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 -- 172.21.15.37:0/631417483 learned_addr learned my addr 172.21.15.37:0/631417483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 0x560a32f10be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32efac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 -- 172.21.15.37:0/631417483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 msgr2=0x560a32f10be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 0x560a32f10be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 -- 172.21.15.37:0/631417483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 msgr2=0x560a32efac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32efac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 -- 172.21.15.37:0/631417483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a32ef8d20 con 0x560a32efb800 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 0x560a32f10be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32efac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8098d60 tx=0x7f6eb809c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c9e6de35821322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6eaaffd640 1 -- 172.21.15.37:0/631417483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb801f040 con 0x560a32efb800 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6eaaffd640 1 -- 172.21.15.37:0/631417483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb80ce330 con 0x560a32efb800 2024-04-25T21:21:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.413+0000 7f6eaaffd640 1 -- 172.21.15.37:0/631417483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80a5340 con 0x560a32efb800 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- 172.21.15.37:0/631417483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 msgr2=0x560a32f11120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8098d60 tx=0x7f6eb809c9c0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- 172.21.15.37:0/631417483 shutdown_connections 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32efb800 0x560a32f11120 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32ef0f40 0x560a32f10be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 --2- 172.21.15.37:0/631417483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32efac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- 172.21.15.37:0/631417483 >> 172.21.15.37:0/631417483 conn(0x560a32eecb30 msgr2=0x560a32eeb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- 172.21.15.37:0/631417483 shutdown_connections 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- 172.21.15.37:0/631417483 wait complete. 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 -- start start 2024-04-25T21:21:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.414+0000 7f6eca21d740 1 Processor -- start 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32f65930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32f709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01af080 con 0x560a32ef0f40 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01ae9b0 con 0x560a32ef0620 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01abc10 con 0x560a32efb800 2024-04-25T21:21:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 10 client.0 _ll_register_callbacks cb 0x7f6eb000bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58832/0 (socket says 172.21.15.37:58832) 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 -- 172.21.15.37:0/318171149 learned_addr learned my addr 172.21.15.37:0/318171149 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32f65930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec59c7640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32f709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 -- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 msgr2=0x560a32f709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32f709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 -- 172.21.15.37:0/318171149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 msgr2=0x560a32f65930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32f65930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 -- 172.21.15.37:0/318171149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a32f70ef0 con 0x560a32ef0f40 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ec49c5640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ba90 tx=0x7f6eb0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75052d5ef9b87abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002a070 con 0x560a32ef0f40 2024-04-25T21:21:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f6eb0004680 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.415+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb80ce330 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6eb003c050 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 10 client.16893 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560a32f72070 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb80ce330 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb0004f40 con 0x560a32ef0f40 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 10 client.16893 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 20 client.16893 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f727e0 0x560a32f74c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 4 client.16893 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a32e5d200 con 0x560a32f727e0 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.416+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f727e0 0x560a32f74c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.417+0000 7f6ec51c6640 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f727e0 0x560a32f74c00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aefa049084a5f71 server_cookie=66f205d41af2dd1f in_seq=0 out_seq=0 2024-04-25T21:21:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.417+0000 7f6ea8ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6ea8ff9640 1 -- 172.21.15.37:0/318171149 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a32e5d200 con 0x560a32f727e0 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6ea8ff9640 10 client.16893 handle_command_reply: tid=0 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 client.16893 shutdown 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 msgr2=0x560a32f65e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ba90 tx=0x7f6eb0011c50 comp rx=0 tx=0).stop 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 shutdown_connections 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560a32f727e0 0x560a32f74c00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560a32efb800 0x560a32f709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560a32ef0f40 0x560a32f65e70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6ea8ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 --2- 172.21.15.37:0/318171149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560a32ef0620 0x560a32f65930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 >> 172.21.15.37:0/318171149 conn(0x560a32eecb30 msgr2=0x560a32eeb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 shutdown_connections 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 1 -- 172.21.15.37:0/318171149 wait complete. 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.431+0000 7f6eca21d740 20 client.16893 trim_cache size 0 max 16384 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:21:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:21:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 msgr2=0x7f6eac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 0x7f6eac0a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc031440 tx=0x7f6ebc005c00 comp rx=0 tx=0).stop 2024-04-25T21:21:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 msgr2=0x7f6ec019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6eb800eaa0 tx=0x7f6eb800ef60 comp rx=0 tx=0).stop 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 shutdown_connections 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ec010c520 0x7f6ec019cfc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f6eac0a3dc0 0x7f6eac0a6280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ec0103b00 0x7f6ec0198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 --2- 172.21.15.37:0/3729546095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ec01031e0 0x7f6ec0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 >> 172.21.15.37:0/3729546095 conn(0x7f6ec0077fd0 msgr2=0x7f6ec00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 shutdown_connections 2024-04-25T21:21:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:26.432+0000 7f6ec742a640 1 -- 172.21.15.37:0/3729546095 wait complete. 2024-04-25T21:21:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:27 smithi177 ceph-mon[33043]: pgmap v406: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:21:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:27 smithi098 ceph-mon[32569]: pgmap v406: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:21:28.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:27 smithi037 ceph-mon[29195]: pgmap v406: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:21:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:29 smithi177 ceph-mon[33043]: pgmap v407: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:21:30.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:29 smithi098 ceph-mon[32569]: pgmap v407: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:21:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:29 smithi037 ceph-mon[29195]: pgmap v407: 97 pgs: 97 active+clean; 2.2 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:21:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:31 smithi177 ceph-mon[33043]: pgmap v408: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T21:21:32.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:31 smithi098 ceph-mon[32569]: pgmap v408: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T21:21:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:31 smithi037 ceph-mon[29195]: pgmap v408: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-25T21:21:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:33 smithi177 ceph-mon[33043]: pgmap v409: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T21:21:34.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:33 smithi098 ceph-mon[32569]: pgmap v409: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T21:21:34.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:33 smithi037 ceph-mon[29195]: pgmap v409: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-25T21:21:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:35 smithi177 ceph-mon[33043]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:21:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:35 smithi098 ceph-mon[32569]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:21:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:35 smithi037 ceph-mon[29195]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T21:21:38.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:37 smithi098 ceph-mon[32569]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:37 smithi037 ceph-mon[29195]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:37 smithi177 ceph-mon[33043]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 -- 172.21.15.37:0/1806644659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 msgr2=0x7fdf201025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 --2- 172.21.15.37:0/1806644659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf201025a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a410 comp rx=0 tx=0).stop 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 -- 172.21.15.37:0/1806644659 shutdown_connections 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 --2- 172.21.15.37:0/1806644659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 0x7fdf2010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 --2- 172.21.15.37:0/1806644659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20102b70 0x7fdf20106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 --2- 172.21.15.37:0/1806644659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf201025a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.532+0000 7fdf25129640 1 -- 172.21.15.37:0/1806644659 >> 172.21.15.37:0/1806644659 conn(0x7fdf200fda40 msgr2=0x7fdf200ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.533+0000 7fdf25129640 1 -- 172.21.15.37:0/1806644659 shutdown_connections 2024-04-25T21:21:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.533+0000 7fdf25129640 1 -- 172.21.15.37:0/1806644659 wait complete. 2024-04-25T21:21:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.533+0000 7fdf25129640 1 Processor -- start 2024-04-25T21:21:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.533+0000 7fdf25129640 1 -- start start 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 0x7fdf20074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20074cd0 0x7fdf201a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20112220 con 0x7fdf201021c0 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201120a0 con 0x7fdf20074cd0 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf25129640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201123a0 con 0x7fdf20107290 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50920/0 (socket says 172.21.15.37:50920) 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 -- 172.21.15.37:0/2313175812 learned_addr learned my addr 172.21.15.37:0/2313175812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20074cd0 0x7fdf201a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf17fff640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 0x7fdf20074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 -- 172.21.15.37:0/2313175812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 msgr2=0x7fdf20074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 0x7fdf20074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 -- 172.21.15.37:0/2313175812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20074cd0 msgr2=0x7fdf201a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20074cd0 0x7fdf201a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 -- 172.21.15.37:0/2313175812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a6fd0 con 0x7fdf201021c0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.534+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005ab0 tx=0x7fdf10003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.535+0000 7fdf1cd99640 1 -- 172.21.15.37:0/2313175812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1003f070 con 0x7fdf201021c0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.535+0000 7fdf1cd99640 1 -- 172.21.15.37:0/2313175812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf10031d60 con 0x7fdf201021c0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.535+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a7260 con 0x7fdf201021c0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.535+0000 7fdf1cd99640 1 -- 172.21.15.37:0/2313175812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10035030 con 0x7fdf201021c0 2024-04-25T21:21:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.535+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a7750 con 0x7fdf201021c0 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.537+0000 7fdf1cd99640 1 -- 172.21.15.37:0/2313175812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf10042070 con 0x7fdf201021c0 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.537+0000 7fdf27e43740 1 Processor -- start 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.537+0000 7fdf27e43740 1 -- start start 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740f40 0x55685374b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x55685374dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1cd99640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 0x7fdf0c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5568536a5970 con 0x556853740620 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5568536424a0 con 0x55685374b800 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf27e43740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5568536a1930 con 0x556853740f40 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50924/0 (socket says 172.21.15.37:50924) 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1f59e640 1 -- 172.21.15.37:0/1364937297 learned_addr learned my addr 172.21.15.37:0/1364937297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 0x7fdf0c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1cd99640 1 -- 172.21.15.37:0/2313175812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fdf100f1060 con 0x7fdf201021c0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf17fff640 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x55685374dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.538+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740f40 0x55685374b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 -- 172.21.15.37:0/1364937297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740f40 msgr2=0x55685374b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740f40 0x55685374b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 -- 172.21.15.37:0/1364937297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 msgr2=0x55685374dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x55685374dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 -- 172.21.15.37:0/1364937297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568533b8210 con 0x556853740620 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdf18012d00 tx=0x7fdf18010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9829daf9675c26b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 0x7fdf0c0a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf18010c40 tx=0x7fdf18017000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf157fa640 1 -- 172.21.15.37:0/1364937297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18024070 con 0x556853740620 2024-04-25T21:21:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf157fa640 1 -- 172.21.15.37:0/1364937297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf18012520 con 0x556853740620 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.539+0000 7fdf157fa640 1 -- 172.21.15.37:0/1364937297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18005280 con 0x556853740620 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- 172.21.15.37:0/1364937297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 msgr2=0x556853740a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fdf18012d00 tx=0x7fdf18010740 comp rx=0 tx=0).stop 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- 172.21.15.37:0/1364937297 shutdown_connections 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x55685374dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740f40 0x55685374b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 --2- 172.21.15.37:0/1364937297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740620 0x556853740a00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- 172.21.15.37:0/1364937297 >> 172.21.15.37:0/1364937297 conn(0x55685373cb30 msgr2=0x55685373cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- 172.21.15.37:0/1364937297 shutdown_connections 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- 172.21.15.37:0/1364937297 wait complete. 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 -- start start 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.540+0000 7fdf27e43740 1 Processor -- start 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x55685374b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x556853748c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55685374e3d0 con 0x556853740f40 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5568536400b0 con 0x55685374b800 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55685374e550 con 0x556853740620 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf27e43740 10 client.0 _ll_register_callbacks cb 0x55685362dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50928/0 (socket says 172.21.15.37:50928) 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 -- 172.21.15.37:0/3263422539 learned_addr learned my addr 172.21.15.37:0/3263422539 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x55685374b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x556853748c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 -- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 msgr2=0x55685374b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x55685374b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 -- 172.21.15.37:0/3263422539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 msgr2=0x556853748c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x556853748c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 -- 172.21.15.37:0/3263422539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556853484620 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x556853748c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdf08012700 tx=0x7fdf08012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d268730545107868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf067fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.541+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004320 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fdf080119a0 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf18012520 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf080044c0 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 10 client.16905 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55685375a410 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf18012520 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf0800f440 con 0x556853740f40 2024-04-25T21:21:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 10 client.16905 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 20 client.16905 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55685375b920 0x55685375dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 4 client.16905 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.542+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568536b3390 con 0x55685375b920 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.543+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55685375b920 0x55685375dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.543+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55685375b920 0x55685375dd10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c0a3981166be03 server_cookie=4931ce39a4ee5acc in_seq=0 out_seq=0 2024-04-25T21:21:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.543+0000 7fdf067fc640 10 client.16905 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf067fc640 1 -- 172.21.15.37:0/3263422539 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568536b3390 con 0x55685375b920 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf067fc640 10 client.16905 handle_command_reply: tid=0 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 client.16905 shutdown 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 msgr2=0x556853748520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fdf08012700 tx=0x7fdf08012bc0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 shutdown_connections 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55685375b920 0x55685375dd10 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55685374b800 0x556853748c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf067fc640 0 client.16905 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x556853748520 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3263422539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x55685374b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 >> 172.21.15.37:0/3263422539 conn(0x55685373cb30 msgr2=0x556853742860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 shutdown_connections 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 1 -- 172.21.15.37:0/3263422539 wait complete. 2024-04-25T21:21:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.544+0000 7fdf27e43740 20 client.16905 trim_cache size 0 max 16384 2024-04-25T21:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 Processor -- start 2024-04-25T21:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 -- start start 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740620 0x55685373c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55685374b800 0x55685374b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201afd80 con 0x556853740f40 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201b18c0 con 0x556853740620 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.559+0000 7fdf27e43740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201b0450 con 0x55685374b800 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50934/0 (socket says 172.21.15.37:50934) 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 -- 172.21.15.37:0/3758936338 learned_addr learned my addr 172.21.15.37:0/3758936338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740620 0x55685373c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55685374b800 0x55685374b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 -- 172.21.15.37:0/3758936338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55685374b800 msgr2=0x55685374b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55685374b800 0x55685374b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 -- 172.21.15.37:0/3758936338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740620 msgr2=0x55685373c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740620 0x55685373c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 -- 172.21.15.37:0/3758936338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556853746a90 con 0x556853740f40 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf17fff640 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800f8e0 tx=0x7fdf08009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e1eee5988f1cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf14ff9640 1 -- 172.21.15.37:0/3758936338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08029070 con 0x556853740f40 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf14ff9640 1 -- 172.21.15.37:0/3758936338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0802a040 con 0x556853740f40 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.560+0000 7fdf14ff9640 1 -- 172.21.15.37:0/3758936338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf08005040 con 0x556853740f40 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- 172.21.15.37:0/3758936338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 msgr2=0x55685374ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800f8e0 tx=0x7fdf08009fd0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- 172.21.15.37:0/3758936338 shutdown_connections 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55685374b800 0x55685374b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556853740f40 0x55685374ad60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 --2- 172.21.15.37:0/3758936338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740620 0x55685373c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- 172.21.15.37:0/3758936338 >> 172.21.15.37:0/3758936338 conn(0x55685373cb30 msgr2=0x556853743af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- 172.21.15.37:0/3758936338 shutdown_connections 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- 172.21.15.37:0/3758936338 wait complete. 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.561+0000 7fdf27e43740 1 -- start start 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 Processor -- start 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x556853745630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740f40 0x556853745b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201b11f0 con 0x55685374b800 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201b0b20 con 0x556853740f40 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201add80 con 0x556853740620 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf27e43740 10 client.0 _ll_register_callbacks cb 0x7fdf08009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50936/0 (socket says 172.21.15.37:50936) 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 -- 172.21.15.37:0/2687525091 learned_addr learned my addr 172.21.15.37:0/2687525091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf17fff640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740f40 0x556853745b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x556853745630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 -- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 msgr2=0x556853745630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x556853745630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 -- 172.21.15.37:0/2687525091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740f40 msgr2=0x556853745b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740f40 0x556853745b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 -- 172.21.15.37:0/2687525091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568537c1390 con 0x55685374b800 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x556853745630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf1f59e640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdf18004980 tx=0x7fdf18012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7b62364aedaaaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.562+0000 7fdf067fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18024040 con 0x55685374b800 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fdf1801b050 con 0x55685374b800 2024-04-25T21:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0802a040 con 0x55685374b800 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf1800ddc0 con 0x55685374b800 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 10 client.16911 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5568537c24e0 con 0x55685374b800 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf0802a040 con 0x55685374b800 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf18004f90 con 0x55685374b800 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 10 client.16911 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 20 client.16911 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5568537c2b10 0x5568537c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 4 client.16911 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568536ad200 con 0x5568537c2b10 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.563+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5568537c2b10 0x5568537c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.564+0000 7fdf1ed9d640 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5568537c2b10 0x5568537c4f30 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41a5e440ddfd82e server_cookie=759ead24b279bc97 in_seq=0 out_seq=0 2024-04-25T21:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.564+0000 7fdf067fc640 10 client.16911 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf067fc640 1 -- 172.21.15.37:0/2687525091 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568536ad200 con 0x5568537c2b10 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf067fc640 10 client.16911 handle_command_reply: tid=0 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 client.16911 shutdown 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 msgr2=0x5568537c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fdf18004980 tx=0x7fdf18012660 comp rx=0 tx=0).stop 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 shutdown_connections 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5568537c2b10 0x5568537c4f30 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.576+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55685374b800 0x5568537c0e50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556853740f40 0x556853745b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf067fc640 0 client.16911 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 1 --2- 172.21.15.37:0/2687525091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556853740620 0x556853745630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 >> 172.21.15.37:0/2687525091 conn(0x55685373cb30 msgr2=0x556853741fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 shutdown_connections 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 1 -- 172.21.15.37:0/2687525091 wait complete. 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf27e43740 20 client.16911 trim_cache size 0 max 16384 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 msgr2=0x7fdf0c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 0x7fdf0c0a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdf18010c40 tx=0x7fdf18017000 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 msgr2=0x7fdf20074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005ab0 tx=0x7fdf10003c80 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 shutdown_connections 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf20074cd0 0x7fdf201a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf0c0a3dc0 0x7fdf0c0a6280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf20107290 0x7fdf20074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 --2- 172.21.15.37:0/2313175812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf201021c0 0x7fdf20074250 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 >> 172.21.15.37:0/2313175812 conn(0x7fdf200fda40 msgr2=0x7fdf200fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 shutdown_connections 2024-04-25T21:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:39.577+0000 7fdf25129640 1 -- 172.21.15.37:0/2313175812 wait complete. 2024-04-25T21:21:40.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:39 smithi098 ceph-mon[32569]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:39 smithi037 ceph-mon[29195]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:39 smithi177 ceph-mon[33043]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-25T21:21:42.169 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:41 smithi098 ceph-mon[32569]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-04-25T21:21:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:41 smithi037 ceph-mon[29195]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-04-25T21:21:42.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:41 smithi177 ceph-mon[33043]: pgmap v413: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.4 KiB/s wr, 15 op/s 2024-04-25T21:21:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:43 smithi098 ceph-mon[32569]: pgmap v414: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T21:21:44.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:43 smithi037 ceph-mon[29195]: pgmap v414: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T21:21:44.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:43 smithi177 ceph-mon[33043]: pgmap v414: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-04-25T21:21:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:45 smithi098 ceph-mon[32569]: pgmap v415: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:21:46.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:45 smithi037 ceph-mon[29195]: pgmap v415: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:21:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:45 smithi177 ceph-mon[33043]: pgmap v415: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-04-25T21:21:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:47 smithi098 ceph-mon[32569]: pgmap v416: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:21:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:21:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:21:48.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:47 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:21:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:47 smithi037 ceph-mon[29195]: pgmap v416: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:21:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:21:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:21:48.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:47 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:21:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:47 smithi177 ceph-mon[33043]: pgmap v416: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:21:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:21:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:21:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:47 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:21:50.141 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:49 smithi177 ceph-mon[33043]: pgmap v417: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:50.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:49 smithi098 ceph-mon[32569]: pgmap v417: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:50.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:49 smithi037 ceph-mon[29195]: pgmap v417: 97 pgs: 97 active+clean; 2.3 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.7 KiB/s wr, 10 op/s 2024-04-25T21:21:50.672 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:21:50.937 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:52.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:51 smithi037 ceph-mon[29195]: pgmap v418: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-25T21:21:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:51 smithi098 ceph-mon[32569]: pgmap v418: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-25T21:21:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:51 smithi177 ceph-mon[33043]: pgmap v418: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 -- 172.21.15.37:0/3138454587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd284101aa0 msgr2=0x7fd284105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 --2- 172.21.15.37:0/3138454587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd284101aa0 0x7fd284105a90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009900 tx=0x7fd26c02a410 comp rx=0 tx=0).stop 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 -- 172.21.15.37:0/3138454587 shutdown_connections 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 --2- 172.21.15.37:0/3138454587 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd284106170 0x7fd28410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 --2- 172.21.15.37:0/3138454587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd284101aa0 0x7fd284105a90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 --2- 172.21.15.37:0/3138454587 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2841010f0 0x7fd2841014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 -- 172.21.15.37:0/3138454587 >> 172.21.15.37:0/3138454587 conn(0x7fd284077fd0 msgr2=0x7fd2840783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 -- 172.21.15.37:0/3138454587 shutdown_connections 2024-04-25T21:21:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.671+0000 7fd28ab37640 1 -- 172.21.15.37:0/3138454587 wait complete. 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 Processor -- start 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 -- start start 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd284106170 0x7fd28410ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd28410f150 0x7fd2841a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841121f0 con 0x7fd2841010f0 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd284112070 con 0x7fd28410f150 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd28ab37640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd284112370 con 0x7fd284106170 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39880/0 (socket says 172.21.15.37:39880) 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 -- 172.21.15.37:0/2134611690 learned_addr learned my addr 172.21.15.37:0/2134611690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd283fff640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd284106170 0x7fd28410ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd28410f150 0x7fd2841a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 -- 172.21.15.37:0/2134611690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd284106170 msgr2=0x7fd28410ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd284106170 0x7fd28410ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 -- 172.21.15.37:0/2134611690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd28410f150 msgr2=0x7fd2841a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd28410f150 0x7fd2841a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.672+0000 7fd2888d3640 1 -- 172.21.15.37:0/2134611690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2841a6f80 con 0x7fd2841010f0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c970 tx=0x7fd27400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd281ffb640 1 -- 172.21.15.37:0/2134611690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274007bb0 con 0x7fd2841010f0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd281ffb640 1 -- 172.21.15.37:0/2134611690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd274007d50 con 0x7fd2841010f0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841a7270 con 0x7fd2841010f0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd281ffb640 1 -- 172.21.15.37:0/2134611690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27400f450 con 0x7fd2841010f0 2024-04-25T21:21:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.673+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841a7790 con 0x7fd2841010f0 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.675+0000 7fd281ffb640 1 -- 172.21.15.37:0/2134611690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd274004ee0 con 0x7fd2841010f0 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 Processor -- start 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 -- start start 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd281ffb640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 0x7fd2700a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490d620 0x5622e490da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 0x5622e49182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd281ffb640 1 -- 172.21.15.37:0/2134611690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fd2740d0580 con 0x7fd2841010f0 2024-04-25T21:21:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 0x7fd2700a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5622e4872970 con 0x5622e4918800 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5622e480f4a0 con 0x5622e490df40 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd28d84e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5622e486e930 con 0x5622e490d620 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd2890d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd2890d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39886/0 (socket says 172.21.15.37:39886) 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd2890d4640 1 -- 172.21.15.37:0/219737766 learned_addr learned my addr 172.21.15.37:0/219737766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd2888d3640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490d620 0x5622e490da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.676+0000 7fd283fff640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 0x5622e49182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 -- 172.21.15.37:0/219737766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490d620 msgr2=0x5622e490da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490d620 0x5622e490da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 -- 172.21.15.37:0/219737766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 msgr2=0x5622e49182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 0x5622e49182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 -- 172.21.15.37:0/219737766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e4585210 con 0x5622e4918800 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd283fff640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 0x5622e49182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 0x7fd2700a6330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd27c015b70 tx=0x7fd27c009d90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd2890d4640 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd27c015fd0 tx=0x7fd27c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e92a91902ed5f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd27affd640 1 -- 172.21.15.37:0/219737766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c012e10 con 0x5622e4918800 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd27affd640 1 -- 172.21.15.37:0/219737766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27c010c10 con 0x5622e4918800 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.677+0000 7fd27affd640 1 -- 172.21.15.37:0/219737766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0053e0 con 0x5622e4918800 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- 172.21.15.37:0/219737766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 msgr2=0x5622e491abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd27c015fd0 tx=0x7fd27c01c000 comp rx=0 tx=0).stop 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- 172.21.15.37:0/219737766 shutdown_connections 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e491abf0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490df40 0x5622e49182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 --2- 172.21.15.37:0/219737766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490d620 0x5622e490da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- 172.21.15.37:0/219737766 >> 172.21.15.37:0/219737766 conn(0x5622e4909b30 msgr2=0x5622e4909f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- 172.21.15.37:0/219737766 shutdown_connections 2024-04-25T21:21:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- 172.21.15.37:0/219737766 wait complete. 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 -- start start 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.678+0000 7fd28d84e740 1 Processor -- start 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e49181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4915490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5622e491b3d0 con 0x5622e4918800 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5622e480d0b0 con 0x5622e490d620 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5622e491b550 con 0x5622e490df40 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd28d84e740 10 client.0 _ll_register_callbacks cb 0x5622e47fad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39890/0 (socket says 172.21.15.37:39890) 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 -- 172.21.15.37:0/2073687894 learned_addr learned my addr 172.21.15.37:0/2073687894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e49181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd283fff640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4915490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 -- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 msgr2=0x5622e4915490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4915490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 -- 172.21.15.37:0/2073687894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 msgr2=0x5622e49181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e49181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 -- 172.21.15.37:0/2073687894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e4651620 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd27c012820 tx=0x7fd27c0151f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f62d3c324f0b00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.679+0000 7fd263fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c0095b0 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd27c010c10 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fd27c005100 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd27c03b050 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 10 client.16923 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e4927380 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27c010c10 con 0x5622e4918800 2024-04-25T21:21:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd27c0296a0 con 0x5622e4918800 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 10 client.16923 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 20 client.16923 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e49287e0 0x5622e492abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 4 client.16923 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.680+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e4880390 con 0x5622e49287e0 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.681+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e49287e0 0x5622e492abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.681+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e49287e0 0x5622e492abd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b892a70c65e85f9 server_cookie=4994a5e8d5df40c7 in_seq=0 out_seq=0 2024-04-25T21:21:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.681+0000 7fd263fff640 10 client.16923 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd263fff640 1 -- 172.21.15.37:0/2073687894 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622e4880390 con 0x5622e49287e0 2024-04-25T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd263fff640 10 client.16923 handle_command_reply: tid=0 2024-04-25T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 client.16923 shutdown 2024-04-25T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 msgr2=0x5622e4915a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd27c012820 tx=0x7fd27c0151f0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 shutdown_connections 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e49287e0 0x5622e492abd0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e4918800 0x5622e4915a30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4915490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd263fff640 0 client.16923 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2073687894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e49181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 >> 172.21.15.37:0/2073687894 conn(0x5622e4909b30 msgr2=0x5622e490f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 shutdown_connections 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 1 -- 172.21.15.37:0/2073687894 wait complete. 2024-04-25T21:21:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.682+0000 7fd28d84e740 20 client.16923 trim_cache size 0 max 16384 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 Processor -- start 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 -- start start 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4910a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e4918800 0x5622e4917cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841afc90 con 0x5622e490d620 2024-04-25T21:21:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841b17d0 con 0x5622e4918800 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd28d84e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841b0360 con 0x5622e490df40 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd2888d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd2888d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39898/0 (socket says 172.21.15.37:39898) 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd2888d3640 1 -- 172.21.15.37:0/2025372341 learned_addr learned my addr 172.21.15.37:0/2025372341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.697+0000 7fd283fff640 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4910a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e4918800 0x5622e4917cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 -- 172.21.15.37:0/2025372341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 msgr2=0x5622e4910a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4910a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 -- 172.21.15.37:0/2025372341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e4918800 msgr2=0x5622e4917cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e4918800 0x5622e4917cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 -- 172.21.15.37:0/2025372341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e490ad60 con 0x5622e490d620 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd274098980 tx=0x7fd274087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f25cc13189635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd27a7fc640 1 -- 172.21.15.37:0/2025372341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740b5070 con 0x5622e490d620 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd27a7fc640 1 -- 172.21.15.37:0/2025372341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd274081e50 con 0x5622e490d620 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.698+0000 7fd27a7fc640 1 -- 172.21.15.37:0/2025372341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27401d040 con 0x5622e490d620 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- 172.21.15.37:0/2025372341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 msgr2=0x5622e4914c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd274098980 tx=0x7fd274087050 comp rx=0 tx=0).stop 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- 172.21.15.37:0/2025372341 shutdown_connections 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e4918800 0x5622e4917cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e490df40 0x5622e4910a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2025372341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490d620 0x5622e4914c40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- 172.21.15.37:0/2025372341 >> 172.21.15.37:0/2025372341 conn(0x5622e4909b30 msgr2=0x5622e4908cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- 172.21.15.37:0/2025372341 shutdown_connections 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- 172.21.15.37:0/2025372341 wait complete. 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 -- start start 2024-04-25T21:21:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.699+0000 7fd28d84e740 1 Processor -- start 2024-04-25T21:21:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e4920740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 0x5622e491be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841b1100 con 0x5622e490df40 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841b0a30 con 0x5622e490d620 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2841adc90 con 0x5622e4918800 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd28d84e740 10 client.0 _ll_register_callbacks cb 0x7fd27c004d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39914/0 (socket says 172.21.15.37:39914) 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 -- 172.21.15.37:0/2140990635 learned_addr learned my addr 172.21.15.37:0/2140990635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e4920740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 0x5622e491be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 -- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 msgr2=0x5622e491be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 0x5622e491be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 -- 172.21.15.37:0/2140990635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 msgr2=0x5622e4920740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.700+0000 7fd283fff640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e4920740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd283fff640 1 -- 172.21.15.37:0/2140990635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e491c650 con 0x5622e490df40 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd2890d4640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 0x5622e491be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e4920740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd283fff640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd26c005ab0 tx=0x7fd26c032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794b1cea7f841ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c035040 con 0x5622e490df40 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fd26c004390 con 0x5622e490df40 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd274081e50 con 0x5622e490df40 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd26c005030 con 0x5622e490df40 2024-04-25T21:21:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd28d84e740 10 client.16929 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e498e6e0 con 0x5622e490df40 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd274081e50 con 0x5622e490df40 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd26c0035f0 con 0x5622e490df40 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.701+0000 7fd28d84e740 10 client.16929 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd28d84e740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd28d84e740 20 client.16929 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e498eee0 0x5622e4991300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd28d84e740 4 client.16929 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622e487a200 con 0x5622e498eee0 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e498eee0 0x5622e4991300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd263fff640 10 client.16929 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.702+0000 7fd2888d3640 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e498eee0 0x5622e4991300 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a97c9199bca3583 server_cookie=40b4c41634993954 in_seq=0 out_seq=0 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.712+0000 7fd263fff640 1 -- 172.21.15.37:0/2140990635 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622e487a200 con 0x5622e498eee0 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.712+0000 7fd263fff640 10 client.16929 handle_command_reply: tid=0 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.712+0000 7fd28d84e740 1 client.16929 shutdown 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 msgr2=0x5622e4920c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fd26c005ab0 tx=0x7fd26c032f70 comp rx=0 tx=0).stop 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 shutdown_connections 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5622e498eee0 0x5622e4991300 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5622e4918800 0x5622e491be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5622e490df40 0x5622e4920c80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 --2- 172.21.15.37:0/2140990635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5622e490d620 0x5622e4920740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd263fff640 0 client.16929 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 >> 172.21.15.37:0/2140990635 conn(0x5622e4909b30 msgr2=0x5622e4908cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 shutdown_connections 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 1 -- 172.21.15.37:0/2140990635 wait complete. 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.713+0000 7fd28d84e740 20 client.16929 trim_cache size 0 max 16384 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:21:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:21:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 msgr2=0x7fd2700a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 0x7fd2700a6330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd27c015b70 tx=0x7fd27c009d90 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 msgr2=0x7fd28410e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c970 tx=0x7fd27400ce30 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 shutdown_connections 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd28410f150 0x7fd2841a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd2700a3e70 0x7fd2700a6330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd284106170 0x7fd28410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 --2- 172.21.15.37:0/2134611690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2841010f0 0x7fd28410e6d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 >> 172.21.15.37:0/2134611690 conn(0x7fd284077fd0 msgr2=0x7fd2840ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 shutdown_connections 2024-04-25T21:21:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:21:52.714+0000 7fd28ab37640 1 -- 172.21.15.37:0/2134611690 wait complete. 2024-04-25T21:21:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 -- 172.21.15.37:0/3724508096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101d10 msgr2=0x7f480c105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 --2- 172.21.15.37:0/3724508096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101d10 0x7f480c105d10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f48000099c0 tx=0x7f4800009cd0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 -- 172.21.15.37:0/3724508096 shutdown_connections 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 --2- 172.21.15.37:0/3724508096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c1063f0 0x7f480c10dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 --2- 172.21.15.37:0/3724508096 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c101360 0x7f480c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 --2- 172.21.15.37:0/3724508096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101d10 0x7f480c105d10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 -- 172.21.15.37:0/3724508096 >> 172.21.15.37:0/3724508096 conn(0x7f480c077fb0 msgr2=0x7f480c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 -- 172.21.15.37:0/3724508096 shutdown_connections 2024-04-25T21:21:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.719+0000 7f48124c0700 1 -- 172.21.15.37:0/3724508096 wait complete. 2024-04-25T21:21:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.720+0000 7f48124c0700 1 Processor -- start 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.720+0000 7f48124c0700 1 -- start start 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c101d10 0x7f480c0fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c1063f0 0x7f480c0fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111af0 con 0x7f480c101360 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111970 con 0x7f480c101d10 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f48124c0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f480c111c70 con 0x7f480c1063f0 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39928/0 (socket says 172.21.15.37:39928) 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 -- 172.21.15.37:0/1303205813 learned_addr learned my addr 172.21.15.37:0/1303205813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480b7fe700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c101d10 0x7f480c0fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f4810a5d700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c1063f0 0x7f480c0fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 -- 172.21.15.37:0/1303205813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c1063f0 msgr2=0x7f480c0fdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c1063f0 0x7f480c0fdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 -- 172.21.15.37:0/1303205813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c101d10 msgr2=0x7f480c0fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c101d10 0x7f480c0fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 -- 172.21.15.37:0/1303205813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480c1a7660 con 0x7f480c101360 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.721+0000 7f480bfff700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e9a0 tx=0x7f47fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.722+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00ca10 con 0x7f480c101360 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.722+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47fc00cbb0 con 0x7f480c101360 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.722+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480c1a7950 con 0x7f480c101360 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.722+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc018680 con 0x7f480c101360 2024-04-25T21:21:52.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.722+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480c1a7e60 con 0x7f480c101360 2024-04-25T21:21:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.723+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f480c1027f0 con 0x7f480c101360 2024-04-25T21:21:52.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.724+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47fc010430 con 0x7f480c101360 2024-04-25T21:21:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.725+0000 7f48097fa700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 0x7f47f40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.725+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f47fc0d0970 con 0x7f480c101360 2024-04-25T21:21:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.725+0000 7f480b7fe700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 0x7f47f40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.725+0000 7f480b7fe700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 0x7f47f40a6270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4800000c00 tx=0x7f4800005fb0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:52.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.730+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47fc09d010 con 0x7f480c101360 2024-04-25T21:21:52.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.986+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f480c1a80d0 con 0x7f480c101360 2024-04-25T21:21:52.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.987+0000 7f48097fa700 1 -- 172.21.15.37:0/1303205813 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f47fc09c910 con 0x7f480c101360 2024-04-25T21:21:52.987 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:21:52.988 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:21:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 msgr2=0x7f47f40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 0x7f47f40a6270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4800000c00 tx=0x7f4800005fb0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 msgr2=0x7f480c0fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00e9a0 tx=0x7f47fc00ecb0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 shutdown_connections 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f47f40a3db0 0x7f47f40a6270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f480c1063f0 0x7f480c0fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c101360 0x7f480c0fd420 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 --2- 172.21.15.37:0/1303205813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f480c101d10 0x7f480c0fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.989+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 >> 172.21.15.37:0/1303205813 conn(0x7f480c077fb0 msgr2=0x7f480c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.990+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 shutdown_connections 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:52.990+0000 7f48124c0700 1 -- 172.21.15.37:0/1303205813 wait complete. 2024-04-25T21:21:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:21:53.644 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id efdf936b --release --await' 2024-04-25T21:21:53.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id efdf936b --release --await 2024-04-25T21:21:53.900 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:53 smithi037 ceph-mon[29195]: pgmap v419: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-04-25T21:21:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:53 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1303205813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:54.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:53 smithi098 ceph-mon[32569]: pgmap v419: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-04-25T21:21:54.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:53 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1303205813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:53 smithi177 ceph-mon[33043]: pgmap v419: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.3 KiB/s wr, 10 op/s 2024-04-25T21:21:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:53 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1303205813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 -- 172.21.15.37:0/3376353049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406b6b0 msgr2=0x7f20c406bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/3376353049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406b6b0 0x7f20c406bb30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f20b4009990 tx=0x7f20b4009ca0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 -- 172.21.15.37:0/3376353049 shutdown_connections 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/3376353049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 0x7f20c4111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/3376353049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c41039c0 0x7f20c406b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/3376353049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c406b6b0 0x7f20c406bb30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 -- 172.21.15.37:0/3376353049 >> 172.21.15.37:0/3376353049 conn(0x7f20c40777b0 msgr2=0x7f20c4077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 -- 172.21.15.37:0/3376353049 shutdown_connections 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.701+0000 7f20cb4ed700 1 -- 172.21.15.37:0/3376353049 wait complete. 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 Processor -- start 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 -- start start 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 0x7f20c419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 0x7f20c419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4114e90 con 0x7f20c41039c0 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4114d10 con 0x7f20c406b6b0 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20cb4ed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20c4115010 con 0x7f20c410f390 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39944/0 (socket says 172.21.15.37:39944) 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 -- 172.21.15.37:0/1885006578 learned_addr learned my addr 172.21.15.37:0/1885006578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 0x7f20c419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c9289700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 0x7f20c419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 -- 172.21.15.37:0/1885006578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 msgr2=0x7f20c419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 0x7f20c419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 -- 172.21.15.37:0/1885006578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 msgr2=0x7f20c419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 0x7f20c419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c8a88700 1 -- 172.21.15.37:0/1885006578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c41a03d0 con 0x7f20c41039c0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.702+0000 7f20c9289700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 0x7f20c419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20c8a88700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20ba7fc700 1 -- 172.21.15.37:0/1885006578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4027070 con 0x7f20c41039c0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20ba7fc700 1 -- 172.21.15.37:0/1885006578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b4022470 con 0x7f20c41039c0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c41a0660 con 0x7f20c41039c0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20ba7fc700 1 -- 172.21.15.37:0/1885006578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4017650 con 0x7f20c41039c0 2024-04-25T21:21:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.703+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c41a7f40 con 0x7f20c41039c0 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.705+0000 7f20ba7fc700 1 -- 172.21.15.37:0/1885006578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20b402a070 con 0x7f20c41039c0 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.706+0000 7f20ba7fc700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 0x7f20b00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.706+0000 7f20ba7fc700 1 -- 172.21.15.37:0/1885006578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f20b40d15e0 con 0x7f20c41039c0 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.706+0000 7f20c9289700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 0x7f20b00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.706+0000 7f20c9289700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 0x7f20b00a62f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0005ee0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 Processor -- start 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 -- start start 2024-04-25T21:21:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4501bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 0x55ee45026ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee44f6ee30 con 0x55ee450271e0 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ee44f15a60 con 0x55ee4501c310 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee44f6adf0 con 0x55ee4501b9f0 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39952/0 (socket says 172.21.15.37:39952) 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 -- 172.21.15.37:0/587672049 learned_addr learned my addr 172.21.15.37:0/587672049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9289700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4501bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c8a88700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 0x55ee45026ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 -- 172.21.15.37:0/587672049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 msgr2=0x55ee4501bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4501bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 -- 172.21.15.37:0/587672049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 msgr2=0x55ee45026ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.707+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 0x55ee45026ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20c9a8a700 1 -- 172.21.15.37:0/587672049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee44ebc9d0 con 0x55ee450271e0 2024-04-25T21:21:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20c8a88700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 0x55ee45026ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0093e0 tx=0x7f20bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7150d2811d60b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20af7fe700 1 -- 172.21.15.37:0/587672049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc019030 con 0x55ee450271e0 2024-04-25T21:21:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20af7fe700 1 -- 172.21.15.37:0/587672049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc0158f0 con 0x55ee450271e0 2024-04-25T21:21:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.708+0000 7f20af7fe700 1 -- 172.21.15.37:0/587672049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0147f0 con 0x55ee450271e0 2024-04-25T21:21:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- 172.21.15.37:0/587672049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 msgr2=0x55ee450295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f20bc0093e0 tx=0x7f20bc009920 comp rx=0 tx=0).stop 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- 172.21.15.37:0/587672049 shutdown_connections 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee450271e0 0x55ee450295d0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4501bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/587672049 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501c310 0x55ee45026ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- 172.21.15.37:0/587672049 >> 172.21.15.37:0/587672049 conn(0x55ee44f49790 msgr2=0x55ee45017ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- 172.21.15.37:0/587672049 shutdown_connections 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- 172.21.15.37:0/587672049 wait complete. 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 -- start start 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.709+0000 7f20d5d76b80 1 Processor -- start 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501b9f0 0x55ee450268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 0x55ee45024120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee45029bd0 con 0x55ee4501c310 2024-04-25T21:21:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ee44f0dcd0 con 0x55ee4501b9f0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee45029d50 con 0x55ee450271e0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20d5d76b80 10 client.0 _ll_register_callbacks cb 0x55ee44f07a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39960/0 (socket says 172.21.15.37:39960) 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 -- 172.21.15.37:0/229332285 learned_addr learned my addr 172.21.15.37:0/229332285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c9289700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501b9f0 0x55ee450268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 0x55ee45024120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 -- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 msgr2=0x55ee45024120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 0x55ee45024120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 -- 172.21.15.37:0/229332285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501b9f0 msgr2=0x55ee450268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501b9f0 0x55ee450268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c8a88700 1 -- 172.21.15.37:0/229332285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee44f26370 con 0x55ee4501c310 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.710+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 0x55ee45024120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20c8a88700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20b40cf330 tx=0x7f20b409d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab141ce6be9d0acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b409da70 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f20b40a5d60 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20bc0158f0 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b409d610 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20d5d76b80 10 client.16947 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee45034370 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20bc0158f0 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f20b40a2b70 con 0x55ee4501c310 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.711+0000 7f20d5d76b80 10 client.16947 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20d5d76b80 10 client.16947 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20d5d76b80 20 client.16947 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee45036ac0 0x55ee45038ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20d5d76b80 4 client.16947 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:21:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee44f7c620 con 0x55ee45036ac0 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20c9289700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee45036ac0 0x55ee45038ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20acff9700 10 client.16947 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.712+0000 7f20c9289700 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee45036ac0 0x55ee45038ee0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21079048eb0573a2 server_cookie=6b06c537e6e26037 in_seq=0 out_seq=0 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20acff9700 1 -- 172.21.15.37:0/229332285 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee44f7c620 con 0x55ee45036ac0 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20acff9700 10 client.16947 handle_command_reply: tid=0 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 client.16947 shutdown 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 msgr2=0x55ee45023be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20b40cf330 tx=0x7f20b409d020 comp rx=0 tx=0).stop 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 shutdown_connections 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee45036ac0 0x55ee45038ee0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee450271e0 0x55ee45024120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee4501b9f0 0x55ee450268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20acff9700 0 client.16947 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/229332285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee45023be0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.713+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 >> 172.21.15.37:0/229332285 conn(0x55ee44f49790 msgr2=0x55ee45017b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.714+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 shutdown_connections 2024-04-25T21:21:55.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.714+0000 7f20d5d76b80 1 -- 172.21.15.37:0/229332285 wait complete. 2024-04-25T21:21:55.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.714+0000 7f20d5d76b80 20 client.16947 trim_cache size 0 max 16384 2024-04-25T21:21:55.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.733+0000 7f20d5d76b80 1 Processor -- start 2024-04-25T21:21:55.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.733+0000 7f20d5d76b80 1 -- start start 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee45022b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4503e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41af3c0 con 0x55ee4501c310 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41b0f00 con 0x55ee450271e0 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41afa90 con 0x55ee4501b9f0 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39968/0 (socket says 172.21.15.37:39968) 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 -- 172.21.15.37:0/2857374201 learned_addr learned my addr 172.21.15.37:0/2857374201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c9289700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee45022b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4503e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 -- 172.21.15.37:0/2857374201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 msgr2=0x55ee45022b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee45022b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 -- 172.21.15.37:0/2857374201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 msgr2=0x55ee4503e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4503e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 -- 172.21.15.37:0/2857374201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee45039560 con 0x55ee4501c310 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c9289700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee45022b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4503e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.734+0000 7f20c8a88700 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f20b4099a70 tx=0x7f20b40a4050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14ad7dc2c1b505a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20aeffd700 1 -- 172.21.15.37:0/2857374201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40a61f0 con 0x55ee4501c310 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20aeffd700 1 -- 172.21.15.37:0/2857374201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b4099040 con 0x55ee4501c310 2024-04-25T21:21:55.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20aeffd700 1 -- 172.21.15.37:0/2857374201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40a5fc0 con 0x55ee4501c310 2024-04-25T21:21:55.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 -- 172.21.15.37:0/2857374201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 msgr2=0x55ee450165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f20b4099a70 tx=0x7f20b40a4050 comp rx=0 tx=0).stop 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 -- 172.21.15.37:0/2857374201 shutdown_connections 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4503e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee45022b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/2857374201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee450165a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 -- 172.21.15.37:0/2857374201 >> 172.21.15.37:0/2857374201 conn(0x55ee44f49790 msgr2=0x55ee45025ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 -- 172.21.15.37:0/2857374201 shutdown_connections 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.735+0000 7f20d5d76b80 1 -- 172.21.15.37:0/2857374201 wait complete. 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 -- start start 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 Processor -- start 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4502f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4502ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41b0830 con 0x55ee4501c310 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41b0160 con 0x55ee450271e0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f20c41ad380 con 0x55ee4501b9f0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20d5d76b80 10 client.0 _ll_register_callbacks cb 0x55ee44f07a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20c8a88700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39974/0 (socket says 172.21.15.37:39974) 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20c8a88700 1 -- 172.21.15.37:0/3626445928 learned_addr learned my addr 172.21.15.37:0/3626445928 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20c9a8a700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4502ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.736+0000 7f20c9289700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4502f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 -- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 msgr2=0x55ee4502f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4502f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 -- 172.21.15.37:0/3626445928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 msgr2=0x55ee4502ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4502ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 -- 172.21.15.37:0/3626445928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4502bfd0 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20c8a88700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f20b40cf330 tx=0x7f20b409d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e6fae07327c9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20acff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b40a61f0 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7f20b40a5d60 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20b4099040 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f20b40b8e60 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20d5d76b80 10 client.16953 fetch_fsmap learned FSMap version 27 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee4502bfd0 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.737+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b4099040 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f20b4099360 con 0x55ee4501c310 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 10 client.16953 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 10 client.16953 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 20 client.16953 populate_metadata read hostname 'smithi037' 2024-04-25T21:21:55.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee4509e040 0x55ee450a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 4 client.16953 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "efdf936b", "release": true, "await": true}] 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "efdf936b", "release": true, "await": true}) -- 0x55ee44f766c0 con 0x55ee4509e040 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20c9289700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee4509e040 0x55ee450a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20acff9700 10 client.16953 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.738+0000 7f20c9289700 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee4509e040 0x55ee450a0430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bbbcc5f1e0a6bc server_cookie=82f0a4b2fdd98699 in_seq=0 out_seq=0 2024-04-25T21:21:55.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20acff9700 1 -- 172.21.15.37:0/3626445928 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ee44f766c0 con 0x55ee4509e040 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20acff9700 10 client.16953 handle_command_reply: tid=0 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 client.16953 shutdown 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 msgr2=0x55ee4502b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f20b40cf330 tx=0x7f20b409d3e0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 shutdown_connections 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee4509e040 0x55ee450a0430 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee450271e0 0x55ee4502ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee4501b9f0 0x55ee4502f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 --2- 172.21.15.37:0/3626445928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee4501c310 0x55ee4502b4d0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 >> 172.21.15.37:0/3626445928 conn(0x55ee44f49790 msgr2=0x55ee45023490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20acff9700 0 client.16953 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 shutdown_connections 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.742+0000 7f20d5d76b80 1 -- 172.21.15.37:0/3626445928 wait complete. 2024-04-25T21:21:55.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20d5d76b80 20 client.16953 trim_cache size 0 max 16384 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 36, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "efdf936b": { 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 36, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:21:55.744 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.5, 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:21:55.745 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 msgr2=0x7f20b00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 0x7f20b00a62f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005fd0 tx=0x7f20c0005ee0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 msgr2=0x7f20c419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f20b4000c00 tx=0x7f20b4009580 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 shutdown_connections 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f20b00a3e30 0x7f20b00a62f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20c410f390 0x7f20c419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20c406b6b0 0x7f20c419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 --2- 172.21.15.37:0/1885006578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c41039c0 0x7f20c419b900 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:55.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 >> 172.21.15.37:0/1885006578 conn(0x7f20c40777b0 msgr2=0x7f20c4100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 shutdown_connections 2024-04-25T21:21:55.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:55.743+0000 7f20cb4ed700 1 -- 172.21.15.37:0/1885006578 wait complete. 2024-04-25T21:21:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:55 smithi037 ceph-mon[29195]: pgmap v420: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:21:56.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:55 smithi098 ceph-mon[32569]: pgmap v420: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:21:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:55 smithi177 ceph-mon[33043]: pgmap v420: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-25T21:21:56.404 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'efdf936b', seconds elapsed: 38.3 2024-04-25T21:21:56.404 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.55 2024-04-25T21:21:56.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:21:56.655 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:58.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:57 smithi037 ceph-mon[29195]: pgmap v421: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:21:58.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:57 smithi098 ceph-mon[32569]: pgmap v421: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:21:58.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:57 smithi177 ceph-mon[33043]: pgmap v421: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 -- 172.21.15.37:0/176046007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64102300 msgr2=0x7eff6410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 --2- 172.21.15.37:0/176046007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64102300 0x7eff6410e7f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b0a0 tx=0x7eff5c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 -- 172.21.15.37:0/176046007 shutdown_connections 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 --2- 172.21.15.37:0/176046007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64102300 0x7eff6410e7f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 --2- 172.21.15.37:0/176046007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64107990 0x7eff64107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 --2- 172.21.15.37:0/176046007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64101940 0x7eff64101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 -- 172.21.15.37:0/176046007 >> 172.21.15.37:0/176046007 conn(0x7eff640788f0 msgr2=0x7eff64078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 -- 172.21.15.37:0/176046007 shutdown_connections 2024-04-25T21:21:58.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.372+0000 7eff6b300700 1 -- 172.21.15.37:0/176046007 wait complete. 2024-04-25T21:21:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 Processor -- start 2024-04-25T21:21:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 -- start start 2024-04-25T21:21:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 0x7eff6419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:58.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64102300 0x7eff6419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111af0 con 0x7eff64107990 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111970 con 0x7eff64101940 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6b300700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7eff64111c70 con 0x7eff64102300 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6989d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6989d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39984/0 (socket says 172.21.15.37:39984) 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6989d700 1 -- 172.21.15.37:0/2806228869 learned_addr learned my addr 172.21.15.37:0/2806228869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6909c700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 0x7eff6419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.373+0000 7eff6889b700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64102300 0x7eff6419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 -- 172.21.15.37:0/2806228869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64102300 msgr2=0x7eff6419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64102300 0x7eff6419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 -- 172.21.15.37:0/2806228869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 msgr2=0x7eff6419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 0x7eff6419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 -- 172.21.15.37:0/2806228869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff641a02b0 con 0x7eff64107990 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6909c700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 0x7eff6419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6989d700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b830 tx=0x7eff5c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c021070 con 0x7eff64107990 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff641a0540 con 0x7eff64107990 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff5c007e30 con 0x7eff64107990 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004700 con 0x7eff64107990 2024-04-25T21:21:58.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.374+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641a7e20 con 0x7eff64107990 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.375+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff64069730 con 0x7eff64107990 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.376+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff5c003d70 con 0x7eff64107990 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.377+0000 7eff5a7fc700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 0x7eff500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.377+0000 7eff6909c700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 0x7eff500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.377+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7eff5c00f020 con 0x7eff64107990 2024-04-25T21:21:58.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.378+0000 7eff6909c700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 0x7eff500a6270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff60005950 tx=0x7eff6000b410 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:21:58.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.381+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff5c02a020 con 0x7eff64107990 2024-04-25T21:21:58.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.637+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff641a7ff0 con 0x7eff64107990 2024-04-25T21:21:58.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.638+0000 7eff5a7fc700 1 -- 172.21.15.37:0/2806228869 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7eff5c09e810 con 0x7eff64107990 2024-04-25T21:21:58.638 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:21:58.638 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 msgr2=0x7eff500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 0x7eff500a6270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff60005950 tx=0x7eff6000b410 comp rx=0 tx=0).stop 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 msgr2=0x7eff6419fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00b830 tx=0x7eff5c004ce0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 shutdown_connections 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7eff500a3db0 0x7eff500a6270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff64107990 0x7eff6419fb70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff64101940 0x7eff6419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 --2- 172.21.15.37:0/2806228869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff64102300 0x7eff6419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 >> 172.21.15.37:0/2806228869 conn(0x7eff640788f0 msgr2=0x7eff6406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 shutdown_connections 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:21:58.640+0000 7eff6b300700 1 -- 172.21.15.37:0/2806228869 wait complete. 2024-04-25T21:21:58.641 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:21:58.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:58 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2806228869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:59.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:58 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2806228869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:59.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:58 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2806228869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:21:59.343 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 86.55 --await-for 42.0' 2024-04-25T21:21:59.343 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 86.55 --await-for 42.0 2024-04-25T21:21:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:21:59.972 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:21:59 smithi037 ceph-mon[29195]: pgmap v422: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:22:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:21:59 smithi098 ceph-mon[32569]: pgmap v422: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:22:00.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:21:59 smithi177 ceph-mon[33043]: pgmap v422: 97 pgs: 97 active+clean; 2.4 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 -- 172.21.15.37:0/1882119341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406b170 msgr2=0x7fce1406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 --2- 172.21.15.37:0/1882119341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406b170 0x7fce1406b5f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fce08009990 tx=0x7fce08009ca0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 -- 172.21.15.37:0/1882119341 shutdown_connections 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 --2- 172.21.15.37:0/1882119341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce1406bb30 0x7fce1410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 --2- 172.21.15.37:0/1882119341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce14108a90 0x7fce14108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 --2- 172.21.15.37:0/1882119341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406b170 0x7fce1406b5f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 -- 172.21.15.37:0/1882119341 >> 172.21.15.37:0/1882119341 conn(0x7fce140786a0 msgr2=0x7fce14078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 -- 172.21.15.37:0/1882119341 shutdown_connections 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 -- 172.21.15.37:0/1882119341 wait complete. 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.296+0000 7fce1a186700 1 Processor -- start 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 -- start start 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce1406b170 0x7fce1419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce14108a90 0x7fce1419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce14112c80 con 0x7fce1406bb30 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce14112b00 con 0x7fce1406b170 2024-04-25T21:22:01.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce1a186700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce14112e00 con 0x7fce14108a90 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60758/0 (socket says 172.21.15.37:60758) 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 -- 172.21.15.37:0/1094691373 learned_addr learned my addr 172.21.15.37:0/1094691373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce137fe700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce1406b170 0x7fce1419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce13fff700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce14108a90 0x7fce1419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 -- 172.21.15.37:0/1094691373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce14108a90 msgr2=0x7fce1419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce14108a90 0x7fce1419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 -- 172.21.15.37:0/1094691373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce1406b170 msgr2=0x7fce1419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce1406b170 0x7fce1419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.297+0000 7fce12ffd700 1 -- 172.21.15.37:0/1094691373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce141a0290 con 0x7fce1406bb30 2024-04-25T21:22:01.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce12ffd700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fce08000c00 tx=0x7fce080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce10ff9700 1 -- 172.21.15.37:0/1094691373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08027070 con 0x7fce1406bb30 2024-04-25T21:22:01.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce141a0520 con 0x7fce1406bb30 2024-04-25T21:22:01.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce10ff9700 1 -- 172.21.15.37:0/1094691373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce080040e0 con 0x7fce1406bb30 2024-04-25T21:22:01.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce10ff9700 1 -- 172.21.15.37:0/1094691373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0800f460 con 0x7fce1406bb30 2024-04-25T21:22:01.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.298+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce141a7e00 con 0x7fce1406bb30 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.300+0000 7fce10ff9700 1 -- 172.21.15.37:0/1094691373 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce0802a070 con 0x7fce1406bb30 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce10ff9700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 0x7fcdfc0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce10ff9700 1 -- 172.21.15.37:0/1094691373 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fce080d17d0 con 0x7fce1406bb30 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce137fe700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 0x7fcdfc0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce137fe700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 0x7fcdfc0a6310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce04009a10 tx=0x7fce04009190 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce24a0fb80 1 Processor -- start 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.301+0000 7fce24a0fb80 1 -- start start 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f779e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f787660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55860f6ccec0 con 0x55860f77a3a0 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55860f673af0 con 0x55860f785270 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55860f6c8e80 con 0x55860f779a80 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60766/0 (socket says 172.21.15.37:60766) 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 -- 172.21.15.37:0/338561541 learned_addr learned my addr 172.21.15.37:0/338561541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce13fff700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f787660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce137fe700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f779e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 -- 172.21.15.37:0/338561541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 msgr2=0x55860f779e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f779e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 -- 172.21.15.37:0/338561541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 msgr2=0x55860f787660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f787660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 -- 172.21.15.37:0/338561541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860f61aa60 con 0x55860f77a3a0 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce137fe700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f779e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce13fff700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f787660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fce12ffd700 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fce08099c60 tx=0x7fce0809d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a669c46dba19880a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.302+0000 7fcdf9ffb700 1 -- 172.21.15.37:0/338561541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce080a40f0 con 0x55860f77a3a0 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fcdf9ffb700 1 -- 172.21.15.37:0/338561541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0809d800 con 0x55860f77a3a0 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fcdf9ffb700 1 -- 172.21.15.37:0/338561541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08022620 con 0x55860f77a3a0 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 -- 172.21.15.37:0/338561541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 msgr2=0x55860f784d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fce08099c60 tx=0x7fce0809d340 comp rx=0 tx=0).stop 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 -- 172.21.15.37:0/338561541 shutdown_connections 2024-04-25T21:22:01.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f787660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f779e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/338561541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f784d30 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 -- 172.21.15.37:0/338561541 >> 172.21.15.37:0/338561541 conn(0x55860f6a7820 msgr2=0x55860f775f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 -- 172.21.15.37:0/338561541 shutdown_connections 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.303+0000 7fce24a0fb80 1 -- 172.21.15.37:0/338561541 wait complete. 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 -- start start 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 Processor -- start 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f784ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f77a3a0 0x55860f781db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55860f6ccec0 con 0x55860f785270 2024-04-25T21:22:01.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55860f66bd60 con 0x55860f77a3a0 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55860f787ba0 con 0x55860f779a80 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce24a0fb80 10 client.0 _ll_register_callbacks cb 0x55860f665ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce13fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce13fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60770/0 (socket says 172.21.15.37:60770) 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.304+0000 7fce13fff700 1 -- 172.21.15.37:0/3777297697 learned_addr learned my addr 172.21.15.37:0/3777297697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce137fe700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f784ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce12ffd700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f77a3a0 0x55860f781db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 -- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 msgr2=0x55860f784ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f784ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 -- 172.21.15.37:0/3777297697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f77a3a0 msgr2=0x55860f781db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f77a3a0 0x55860f781db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 -- 172.21.15.37:0/3777297697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860f684400 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce13fff700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fce0c010b30 tx=0x7fce0c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5365e00fbd1c50c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fcdf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c01a020 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fce0c00ea60 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce0809d800 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce0c025050 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce24a0fb80 10 client.16971 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.305+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55860f792230 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce0809d800 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fce0c015cb0 con 0x55860f785270 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 10 client.16971 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:01.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 10 client.16971 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 20 client.16971 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f796cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 4 client.16971 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55860f6da6b0 con 0x55860f7948c0 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.306+0000 7fce137fe700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f796cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fce137fe700 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f796cb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964159ae5fca9d9c server_cookie=4aa34d8423e04750 in_seq=0 out_seq=0 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fcdf2ffd700 10 client.16971 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3777297697 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55860f6da6b0 con 0x55860f7948c0 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fcdf2ffd700 10 client.16971 handle_command_reply: tid=0 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fce24a0fb80 1 client.16971 shutdown 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 msgr2=0x55860f7822f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.307+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fce0c010b30 tx=0x7fce0c010ef0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 shutdown_connections 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f796cb0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fcdf2ffd700 0 client.16971 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f7822f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f784ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3777297697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f77a3a0 0x55860f781db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 >> 172.21.15.37:0/3777297697 conn(0x55860f6a7820 msgr2=0x55860f775b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 shutdown_connections 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3777297697 wait complete. 2024-04-25T21:22:01.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.308+0000 7fce24a0fb80 20 client.16971 trim_cache size 0 max 16384 2024-04-25T21:22:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.327+0000 7fce24a0fb80 1 Processor -- start 2024-04-25T21:22:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.327+0000 7fce24a0fb80 1 -- start start 2024-04-25T21:22:01.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f776050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f7845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce141af2f0 con 0x55860f77a3a0 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce141b0e30 con 0x55860f785270 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce141af9c0 con 0x55860f779a80 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60786/0 (socket says 172.21.15.37:60786) 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 -- 172.21.15.37:0/217810745 learned_addr learned my addr 172.21.15.37:0/217810745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce137fe700 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f776050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce13fff700 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f7845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 -- 172.21.15.37:0/217810745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 msgr2=0x55860f776050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f776050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 -- 172.21.15.37:0/217810745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 msgr2=0x55860f7845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f7845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 -- 172.21.15.37:0/217810745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55860f780320 con 0x55860f77a3a0 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fce12ffd700 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fce080cefd0 tx=0x7fce0809d260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0593c4c03a903a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fcdf97fa700 1 -- 172.21.15.37:0/217810745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce080a1bb0 con 0x55860f77a3a0 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fcdf97fa700 1 -- 172.21.15.37:0/217810745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce080a40c0 con 0x55860f77a3a0 2024-04-25T21:22:01.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.328+0000 7fcdf97fa700 1 -- 172.21.15.37:0/217810745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0809dcc0 con 0x55860f77a3a0 2024-04-25T21:22:01.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 -- 172.21.15.37:0/217810745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 msgr2=0x55860f7907a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fce080cefd0 tx=0x7fce0809d260 comp rx=0 tx=0).stop 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 -- 172.21.15.37:0/217810745 shutdown_connections 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f785270 0x55860f7845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f779a80 0x55860f776050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/217810745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f77a3a0 0x55860f7907a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 -- 172.21.15.37:0/217810745 >> 172.21.15.37:0/217810745 conn(0x55860f6a7820 msgr2=0x55860f790090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 -- 172.21.15.37:0/217810745 shutdown_connections 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.329+0000 7fce24a0fb80 1 -- 172.21.15.37:0/217810745 wait complete. 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 -- start start 2024-04-25T21:22:01.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 Processor -- start 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f779a80 0x55860f78d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f77a3a0 0x55860f78b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce141b0760 con 0x55860f785270 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce141b0090 con 0x55860f779a80 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fce141ad2b0 con 0x55860f77a3a0 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce24a0fb80 10 client.0 _ll_register_callbacks cb 0x55860f665ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce13fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce13fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60790/0 (socket says 172.21.15.37:60790) 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce13fff700 1 -- 172.21.15.37:0/3429986662 learned_addr learned my addr 172.21.15.37:0/3429986662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:01.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce12ffd700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f77a3a0 0x55860f78b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce137fe700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f779a80 0x55860f78d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce13fff700 1 -- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f77a3a0 msgr2=0x55860f78b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.330+0000 7fce13fff700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f77a3a0 0x55860f78b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce13fff700 1 -- 172.21.15.37:0/3429986662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f779a80 msgr2=0x55860f78d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce13fff700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f779a80 0x55860f78d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce13fff700 1 -- 172.21.15.37:0/3429986662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55860f7fadc0 con 0x55860f785270 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce13fff700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fce0c016650 tx=0x7fce0c007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3315226514b657e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0c01a070 con 0x55860f785270 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fce0c00e860 con 0x55860f785270 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce080a40c0 con 0x55860f785270 2024-04-25T21:22:01.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fce0c00ea60 con 0x55860f785270 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 10 client.16977 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55860f7fbfe0 con 0x55860f785270 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce080a40c0 con 0x55860f785270 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fce0c005150 con 0x55860f785270 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 10 client.16977 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 10 client.16977 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 20 client.16977 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.331+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f7fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.332+0000 7fce24a0fb80 4 client.16977 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.55, "await_for": 42.0}] 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.332+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.55, "await_for": 42.0}) -- 0x55860f6d4750 con 0x55860f7948c0 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.332+0000 7fce137fe700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f7fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:01.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.332+0000 7fce137fe700 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f7fe6b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c81fe9002d806fd server_cookie=f9eaa256cee944a3 in_seq=0 out_seq=0 2024-04-25T21:22:01.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.332+0000 7fcdf2ffd700 10 client.16977 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fcdf2ffd700 1 -- 172.21.15.37:0/3429986662 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55860f6d4750 con 0x55860f7948c0 2024-04-25T21:22:01.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fcdf2ffd700 10 client.16977 handle_command_reply: tid=0 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 client.16977 shutdown 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 msgr2=0x55860f78bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fce0c016650 tx=0x7fce0c007c60 comp rx=0 tx=0).stop 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 shutdown_connections 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55860f7948c0 0x55860f7fe6b0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55860f785270 0x55860f78bd70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55860f779a80 0x55860f78d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 --2- 172.21.15.37:0/3429986662 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55860f77a3a0 0x55860f78b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 >> 172.21.15.37:0/3429986662 conn(0x55860f6a7820 msgr2=0x55860f7907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fcdf2ffd700 0 client.16977 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 shutdown_connections 2024-04-25T21:22:01.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 1 -- 172.21.15.37:0/3429986662 wait complete. 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.346+0000 7fce24a0fb80 20 client.16977 trim_cache size 0 max 16384 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 38, 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "9009a9a9": { 2024-04-25T21:22:01.348 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 38, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.5, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:22:01.349 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:01.350 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 msgr2=0x7fcdfc0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 0x7fcdfc0a6310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fce04009a10 tx=0x7fce04009190 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 msgr2=0x7fce1419b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fce08000c00 tx=0x7fce080047b0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 shutdown_connections 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fcdfc0a3e50 0x7fcdfc0a6310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce14108a90 0x7fce1419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce1406b170 0x7fce1419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 --2- 172.21.15.37:0/1094691373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1406bb30 0x7fce1419b7c0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 >> 172.21.15.37:0/1094691373 conn(0x7fce140786a0 msgr2=0x7fce1410dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 shutdown_connections 2024-04-25T21:22:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:01.347+0000 7fce1a186700 1 -- 172.21.15.37:0/1094691373 wait complete. 2024-04-25T21:22:02.070 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:01 smithi037 ceph-mon[29195]: pgmap v423: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:22:02.100 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9009a9a9', quiesce took 5.7 seconds. Will release after: 26.55 2024-04-25T21:22:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:01 smithi098 ceph-mon[32569]: pgmap v423: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:22:02.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:01 smithi177 ceph-mon[33043]: pgmap v423: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-25T21:22:04.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:03 smithi098 ceph-mon[32569]: pgmap v424: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-25T21:22:04.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:03 smithi037 ceph-mon[29195]: pgmap v424: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-25T21:22:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:03 smithi177 ceph-mon[33043]: pgmap v424: 97 pgs: 97 active+clean; 2.5 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-25T21:22:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 -- 172.21.15.37:0/3200128784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101a90 msgr2=0x7fecc4105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 --2- 172.21.15.37:0/3200128784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101a90 0x7fecc4105ad0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac02a410 comp rx=0 tx=0).stop 2024-04-25T21:22:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 -- 172.21.15.37:0/3200128784 shutdown_connections 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 --2- 172.21.15.37:0/3200128784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc41061b0 0x7fecc410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 --2- 172.21.15.37:0/3200128784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc4101a90 0x7fecc4105ad0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 --2- 172.21.15.37:0/3200128784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41010e0 0x7fecc41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 -- 172.21.15.37:0/3200128784 >> 172.21.15.37:0/3200128784 conn(0x7fecc40771d0 msgr2=0x7fecc40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 -- 172.21.15.37:0/3200128784 shutdown_connections 2024-04-25T21:22:05.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.816+0000 7fecc9220640 1 -- 172.21.15.37:0/3200128784 wait complete. 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 Processor -- start 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 -- start start 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41061b0 0x7fecc4198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 0x7fecc419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41121f0 con 0x7fecc41010e0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4112070 con 0x7fecc419cf10 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc9220640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4112370 con 0x7fecc41061b0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.817+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60800/0 (socket says 172.21.15.37:60800) 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 -- 172.21.15.37:0/1311430978 learned_addr learned my addr 172.21.15.37:0/1311430978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecbbfff640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41061b0 0x7fecc4198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 -- 172.21.15.37:0/1311430978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41061b0 msgr2=0x7fecc4198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41061b0 0x7fecc4198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc359e640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 0x7fecc419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 -- 172.21.15.37:0/1311430978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 msgr2=0x7fecc419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 0x7fecc419d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 -- 172.21.15.37:0/1311430978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc419da00 con 0x7fecc41010e0 2024-04-25T21:22:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc359e640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 0x7fecc419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fecb400d950 tx=0x7fecb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc0d99640 1 -- 172.21.15.37:0/1311430978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4017030 con 0x7fecc41010e0 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc0d99640 1 -- 172.21.15.37:0/1311430978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4004020 con 0x7fecc41010e0 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc0d99640 1 -- 172.21.15.37:0/1311430978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40047a0 con 0x7fecc41010e0 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc419dcf0 con 0x7fecc41010e0 2024-04-25T21:22:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.818+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc41a54b0 con 0x7fecc41010e0 2024-04-25T21:22:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.821+0000 7fecc0d99640 1 -- 172.21.15.37:0/1311430978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecb4009a30 con 0x7fecc41010e0 2024-04-25T21:22:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.821+0000 7feccc010740 1 Processor -- start 2024-04-25T21:22:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.821+0000 7feccc010740 1 -- start start 2024-04-25T21:22:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.821+0000 7feccc010740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7feccc010740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7feccc010740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7feccc010740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633f692e970 con 0x5633f69cda30 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7feccc010740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5633f68cb4a0 con 0x5633f69ce350 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7feccc010740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5633f692a930 con 0x5633f69d8c10 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60806/0 (socket says 172.21.15.37:60806) 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 -- 172.21.15.37:0/1981566026 learned_addr learned my addr 172.21.15.37:0/1981566026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc0d99640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 0x7fecb00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecbbfff640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 0x7fecb00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 -- 172.21.15.37:0/1981566026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 msgr2=0x5633f69db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc0d99640 1 -- 172.21.15.37:0/1311430978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fecb40d02e0 con 0x7fecc41010e0 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 -- 172.21.15.37:0/1981566026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 msgr2=0x5633f69d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 -- 172.21.15.37:0/1981566026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f6641210 con 0x5633f69cda30 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.822+0000 7fecc359e640 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0139e0 tx=0x7fecbc013ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d97189ebf478f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7fecb97fa640 1 -- 172.21.15.37:0/1981566026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc01fb80 con 0x5633f69cda30 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7fecb97fa640 1 -- 172.21.15.37:0/1981566026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc01fd20 con 0x5633f69cda30 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7fecb97fa640 1 -- 172.21.15.37:0/1981566026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0273c0 con 0x5633f69cda30 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 -- 172.21.15.37:0/1981566026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 msgr2=0x5633f69cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0139e0 tx=0x7fecbc013ea0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7fecc359e640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 0x7fecb00a6280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fecbc022000 tx=0x7fecbc020000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 -- 172.21.15.37:0/1981566026 shutdown_connections 2024-04-25T21:22:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 --2- 172.21.15.37:0/1981566026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69cde10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.823+0000 7feccc010740 1 -- 172.21.15.37:0/1981566026 >> 172.21.15.37:0/1981566026 conn(0x5633f69c9f40 msgr2=0x5633f69ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- 172.21.15.37:0/1981566026 shutdown_connections 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- 172.21.15.37:0/1981566026 wait complete. 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- start start 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 Processor -- start 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633f69db840 con 0x5633f69cda30 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5633f68c90b0 con 0x5633f69ce350 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5633f69db9c0 con 0x5633f69d8c10 2024-04-25T21:22:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7feccc010740 10 client.0 _ll_register_callbacks cb 0x5633f68b6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60818/0 (socket says 172.21.15.37:60818) 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7fecc2d9d640 1 -- 172.21.15.37:0/4231419753 learned_addr learned my addr 172.21.15.37:0/4231419753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.824+0000 7fecc359e640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecbbfff640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 -- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 msgr2=0x5633f69d5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69d5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 -- 172.21.15.37:0/4231419753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 msgr2=0x5633f69d5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 -- 172.21.15.37:0/4231419753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f670d620 con 0x5633f69cda30 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc359e640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fecb40cda30 tx=0x7fecb406abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda58413c4715a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb409c220 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fecb4098890 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecbc01fd20 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecb40a05d0 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7feccc010740 10 client.16989 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633f69e7790 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc01fd20 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.825+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecb40a0160 con 0x5633f69cda30 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 10 client.16989 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 20 client.16989 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f69e8c00 0x5633f69eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 4 client.16989 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f693c390 con 0x5633f69e8c00 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7fecbbfff640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f69e8c00 0x5633f69eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7fecbbfff640 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f69e8c00 0x5633f69eaff0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6ec575bb1346a3 server_cookie=79efdf1afe831420 in_seq=0 out_seq=0 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.826+0000 7fecaa7fc640 10 client.16989 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7fecaa7fc640 1 -- 172.21.15.37:0/4231419753 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633f693c390 con 0x5633f69e8c00 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7fecaa7fc640 10 client.16989 handle_command_reply: tid=0 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 client.16989 shutdown 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 msgr2=0x5633f69d85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fecb40cda30 tx=0x7fecb406abd0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 shutdown_connections 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f69e8c00 0x5633f69eaff0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69d8c10 0x5633f69d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69d5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 --2- 172.21.15.37:0/4231419753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69cda30 0x5633f69d85a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 >> 172.21.15.37:0/4231419753 conn(0x5633f69c9f40 msgr2=0x5633f69cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7fecaa7fc640 0 client.16989 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 shutdown_connections 2024-04-25T21:22:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.827+0000 7feccc010740 1 -- 172.21.15.37:0/4231419753 wait complete. 2024-04-25T21:22:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.828+0000 7feccc010740 20 client.16989 trim_cache size 0 max 16384 2024-04-25T21:22:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 Processor -- start 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 -- start start 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69cda30 0x5633f69d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ab960 con 0x5633f69d8c10 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ad4a0 con 0x5633f69ce350 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7feccc010740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ac030 con 0x5633f69cda30 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60822/0 (socket says 172.21.15.37:60822) 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 -- 172.21.15.37:0/3261502947 learned_addr learned my addr 172.21.15.37:0/3261502947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69cda30 0x5633f69d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecbbfff640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 -- 172.21.15.37:0/3261502947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69cda30 msgr2=0x5633f69d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69cda30 0x5633f69d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 -- 172.21.15.37:0/3261502947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 msgr2=0x5633f69e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69e0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecc359e640 1 -- 172.21.15.37:0/3261502947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f69d6180 con 0x5633f69d8c10 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.843+0000 7fecbbfff640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.844+0000 7fecc359e640 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0092c0 tx=0x7fecbc0117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc41f7322cd0646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.844+0000 7fecb8ff9640 1 -- 172.21.15.37:0/3261502947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc038070 con 0x5633f69d8c10 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.844+0000 7fecb8ff9640 1 -- 172.21.15.37:0/3261502947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc003970 con 0x5633f69d8c10 2024-04-25T21:22:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.844+0000 7fecb8ff9640 1 -- 172.21.15.37:0/3261502947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc027900 con 0x5633f69d8c10 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- 172.21.15.37:0/3261502947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 msgr2=0x5633f69e1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0092c0 tx=0x7fecbc0117d0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- 172.21.15.37:0/3261502947 shutdown_connections 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69e1150 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69ce350 0x5633f69e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- 172.21.15.37:0/3261502947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69cda30 0x5633f69d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- 172.21.15.37:0/3261502947 >> 172.21.15.37:0/3261502947 conn(0x5633f69c9f40 msgr2=0x5633f69e36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- 172.21.15.37:0/3261502947 shutdown_connections 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- 172.21.15.37:0/3261502947 wait complete. 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- start start 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 Processor -- start 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 0x5633f69e08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69ce350 0x5633f69e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41acdd0 con 0x5633f69d8c10 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ac700 con 0x5633f69cda30 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41a9960 con 0x5633f69ce350 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.845+0000 7feccc010740 10 client.0 _ll_register_callbacks cb 0x7fecb409be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60834/0 (socket says 172.21.15.37:60834) 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 -- 172.21.15.37:0/1281563989 learned_addr learned my addr 172.21.15.37:0/1281563989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecbbfff640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69ce350 0x5633f69e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 0x5633f69e08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 -- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69ce350 msgr2=0x5633f69e0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69ce350 0x5633f69e0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 -- 172.21.15.37:0/1281563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 msgr2=0x5633f69e08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 0x5633f69e08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 -- 172.21.15.37:0/1281563989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f69df460 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 0x5633f69e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecc359e640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0095d0 tx=0x7fecbc0273c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e524c6460e2e9e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecaa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc038040 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8513+0+0 (secure 0 0 0) 0x7fecbc003d20 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecbc003970 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.846+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fecbc0135c0 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 10 client.16995 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633f6a4fa60 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc003970 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fecbc027ce0 con 0x5633f69d8c10 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 10 client.16995 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 20 client.16995 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f6a4fed0 0x5633f6a522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 4 client.16995 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633f6936200 con 0x5633f6a4fed0 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.847+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f6a4fed0 0x5633f6a522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.848+0000 7fecc2d9d640 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f6a4fed0 0x5633f6a522f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2aaf49080935751 server_cookie=707552ef0b5c01af in_seq=0 out_seq=0 2024-04-25T21:22:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.848+0000 7fecaa7fc640 10 client.16995 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7fecaa7fc640 1 -- 172.21.15.37:0/1281563989 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633f6936200 con 0x5633f6a4fed0 2024-04-25T21:22:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7fecaa7fc640 10 client.16995 handle_command_reply: tid=0 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 client.16995 shutdown 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 msgr2=0x5633f69ded70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0095d0 tx=0x7fecbc0273c0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 shutdown_connections 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5633f6a4fed0 0x5633f6a522f0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633f69d8c10 0x5633f69ded70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7fecaa7fc640 0 client.16995 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633f69ce350 0x5633f69e0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 --2- 172.21.15.37:0/1281563989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5633f69cda30 0x5633f69e08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 >> 172.21.15.37:0/1281563989 conn(0x5633f69c9f40 msgr2=0x5633f69e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 shutdown_connections 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.860+0000 7feccc010740 1 -- 172.21.15.37:0/1281563989 wait complete. 2024-04-25T21:22:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7feccc010740 20 client.16995 trim_cache size 0 max 16384 2024-04-25T21:22:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:22:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:22:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:22:05.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 msgr2=0x7fecb00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 0x7fecb00a6280 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fecbc022000 tx=0x7fecbc020000 comp rx=0 tx=0).stop 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 msgr2=0x7fecc4198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fecb400d950 tx=0x7fecb400de10 comp rx=0 tx=0).stop 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 shutdown_connections 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fecc419cf10 0x7fecc419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fecb00a3dc0 0x7fecb00a6280 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fecc41061b0 0x7fecc4198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 --2- 172.21.15.37:0/1311430978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc41010e0 0x7fecc4198650 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 >> 172.21.15.37:0/1311430978 conn(0x7fecc40771d0 msgr2=0x7fecc40fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 shutdown_connections 2024-04-25T21:22:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:05.861+0000 7fecc9220640 1 -- 172.21.15.37:0/1311430978 wait complete. 2024-04-25T21:22:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:05 smithi098 ceph-mon[32569]: pgmap v425: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:22:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:05 smithi037 ceph-mon[29195]: pgmap v425: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:22:06.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:05 smithi177 ceph-mon[33043]: pgmap v425: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 69 KiB/s wr, 34 op/s 2024-04-25T21:22:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:07 smithi098 ceph-mon[32569]: pgmap v426: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-04-25T21:22:08.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:07 smithi098 ceph-mon[32569]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:22:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:07 smithi037 ceph-mon[29195]: pgmap v426: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-04-25T21:22:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:07 smithi037 ceph-mon[29195]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:22:08.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:07 smithi177 ceph-mon[33043]: pgmap v426: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 66 KiB/s wr, 28 op/s 2024-04-25T21:22:08.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:07 smithi177 ceph-mon[33043]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:22:10.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:09 smithi098 ceph-mon[32569]: pgmap v428: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-04-25T21:22:10.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:09 smithi037 ceph-mon[29195]: pgmap v428: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-04-25T21:22:10.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:09 smithi177 ceph-mon[33043]: pgmap v428: 97 pgs: 97 active+clean; 2.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-04-25T21:22:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:11 smithi098 ceph-mon[32569]: pgmap v429: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:12.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:11 smithi037 ceph-mon[29195]: pgmap v429: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:12.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:11 smithi177 ceph-mon[33043]: pgmap v429: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:14.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:13 smithi098 ceph-mon[32569]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:13 smithi037 ceph-mon[29195]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:13 smithi177 ceph-mon[33043]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-25T21:22:15.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:14 smithi037 ceph-mon[29195]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:15.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:14 smithi177 ceph-mon[33043]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:15.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:14 smithi098 ceph-mon[32569]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:16 smithi177 ceph-mon[33043]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T21:22:16.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:16 smithi098 ceph-mon[32569]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T21:22:16.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:16 smithi037 ceph-mon[29195]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T21:22:17.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:17 smithi177 ceph-mon[33043]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9 KiB/s wr, 14 op/s 2024-04-25T21:22:17.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:17 smithi098 ceph-mon[32569]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9 KiB/s wr, 14 op/s 2024-04-25T21:22:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:17 smithi037 ceph-mon[29195]: pgmap v433: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9 KiB/s wr, 14 op/s 2024-04-25T21:22:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 -- 172.21.15.37:0/3487991465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 msgr2=0x7fb8d010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 --2- 172.21.15.37:0/3487991465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d010ee70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009ca0 tx=0x7fb8c802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 -- 172.21.15.37:0/3487991465 shutdown_connections 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 --2- 172.21.15.37:0/3487991465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d010ee70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 --2- 172.21.15.37:0/3487991465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d0102b80 0x7fb8d0106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 --2- 172.21.15.37:0/3487991465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 0x7fb8d01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 -- 172.21.15.37:0/3487991465 >> 172.21.15.37:0/3487991465 conn(0x7fb8d00793e0 msgr2=0x7fb8d00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 -- 172.21.15.37:0/3487991465 shutdown_connections 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.961+0000 7fb8d7720640 1 -- 172.21.15.37:0/3487991465 wait complete. 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 Processor -- start 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 -- start start 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 0x7fb8d0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d019cf30 0x7fb8d019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112210 con 0x7fb8d01072a0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112090 con 0x7fb8d01021d0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.962+0000 7fb8d7720640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0112390 con 0x7fb8d019cf30 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35498/0 (socket says 172.21.15.37:35498) 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/314341042 learned_addr learned my addr 172.21.15.37:0/314341042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d019cf30 0x7fb8d019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 0x7fb8d0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/314341042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d019cf30 msgr2=0x7fb8d019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d019cf30 0x7fb8d019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/314341042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 msgr2=0x7fb8d0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 0x7fb8d0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/314341042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d019da20 con 0x7fb8d01072a0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d740 tx=0x7fb8c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8c67fc640 1 -- 172.21.15.37:0/314341042 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0017070 con 0x7fb8d01072a0 2024-04-25T21:22:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8c67fc640 1 -- 172.21.15.37:0/314341042 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c000de50 con 0x7fb8d01072a0 2024-04-25T21:22:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d019dd10 con 0x7fb8d01072a0 2024-04-25T21:22:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8c67fc640 1 -- 172.21.15.37:0/314341042 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0004ce0 con 0x7fb8d01072a0 2024-04-25T21:22:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.963+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a54b0 con 0x7fb8d01072a0 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.966+0000 7fb8c67fc640 1 -- 172.21.15.37:0/314341042 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb8c0004e80 con 0x7fb8d01072a0 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.966+0000 7fb8da43a740 1 Processor -- start 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.966+0000 7fb8da43a740 1 -- start start 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68620 0x563fa2c68a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 0x563fa2c75bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563fa2bcd970 con 0x563fa2c73800 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563fa2b6a4a0 con 0x563fa2c68f40 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8da43a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563fa2bc9930 con 0x563fa2c68620 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8c67fc640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 0x7fb8bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 0x563fa2c75bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 0x563fa2c75bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35514/0 (socket says 172.21.15.37:35514) 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/1180385956 learned_addr learned my addr 172.21.15.37:0/1180385956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 0x7fb8bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68620 0x563fa2c68a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8c67fc640 1 -- 172.21.15.37:0/314341042 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb8c00d0130 con 0x7fb8d01072a0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1180385956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68620 msgr2=0x563fa2c68a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68620 0x563fa2c68a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1180385956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 msgr2=0x563fa2c75bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 0x563fa2c75bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.967+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1180385956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa28e0210 con 0x563fa2c68f40 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 0x7fb8bc0a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8006010 tx=0x7fb8c8009a20 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c732c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009b70 tx=0x7fb8cc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef5bb99d208bbfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8bb7fe640 1 -- 172.21.15.37:0/1180385956 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc009e60 con 0x563fa2c68f40 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8bb7fe640 1 -- 172.21.15.37:0/1180385956 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8cc012900 con 0x563fa2c68f40 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8bb7fe640 1 -- 172.21.15.37:0/1180385956 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00fa70 con 0x563fa2c68f40 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8da43a740 1 -- 172.21.15.37:0/1180385956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 msgr2=0x563fa2c732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.968+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c732c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009b70 tx=0x7fb8cc0117b0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- 172.21.15.37:0/1180385956 shutdown_connections 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c73800 0x563fa2c75bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c732c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1180385956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68620 0x563fa2c68a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- 172.21.15.37:0/1180385956 >> 172.21.15.37:0/1180385956 conn(0x563fa2c64b30 msgr2=0x563fa2c64f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- 172.21.15.37:0/1180385956 shutdown_connections 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- 172.21.15.37:0/1180385956 wait complete. 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- start start 2024-04-25T21:22:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 Processor -- start 2024-04-25T21:22:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 0x563fa2c73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c70940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.969+0000 7fb8da43a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563fa2c76430 con 0x563fa2c68f40 2024-04-25T21:22:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8da43a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563fa2b680b0 con 0x563fa2c68620 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8da43a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x563fa2c765b0 con 0x563fa2c73800 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8da43a740 10 client.0 _ll_register_callbacks cb 0x563fa2b55d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35524/0 (socket says 172.21.15.37:35524) 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/3286873966 learned_addr learned my addr 172.21.15.37:0/3286873966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 0x563fa2c73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c70940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 msgr2=0x563fa2c70940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c70940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/3286873966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 msgr2=0x563fa2c73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 0x563fa2c73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 -- 172.21.15.37:0/3286873966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa29ac620 con 0x563fa2c68f40 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 0x563fa2c73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:22:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb8c009bdd0 tx=0x7fb8c009baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb5e84a3875c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0020040 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb8c00cdb00 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.970+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc012900 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8c00a0480 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 10 client.17007 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fa2c82360 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8cc012900 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb8c00a4940 con 0x563fa2c68f40 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 10 client.17007 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 20 client.17007 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2c837d0 0x563fa2c85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 4 client.17007 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa2bdb390 con 0x563fa2c837d0 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.971+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2c837d0 0x563fa2c85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.972+0000 7fb8b8ff9640 10 client.17007 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.972+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2c837d0 0x563fa2c85c00 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a6d3e420aa578c server_cookie=d6c96e7d29c79596 in_seq=0 out_seq=0 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/3286873966 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fa2bdb390 con 0x563fa2c837d0 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8b8ff9640 10 client.17007 handle_command_reply: tid=0 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 client.17007 shutdown 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 msgr2=0x563fa2c70400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb8c009bdd0 tx=0x7fb8c009baf0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 shutdown_connections 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2c837d0 0x563fa2c85c00 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c70940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68f40 0x563fa2c70400 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8b8ff9640 0 client.17007 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 --2- 172.21.15.37:0/3286873966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68620 0x563fa2c73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 >> 172.21.15.37:0/3286873966 conn(0x563fa2c64b30 msgr2=0x563fa2c6a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 shutdown_connections 2024-04-25T21:22:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 1 -- 172.21.15.37:0/3286873966 wait complete. 2024-04-25T21:22:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.973+0000 7fb8da43a740 20 client.17007 trim_cache size 0 max 16384 2024-04-25T21:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 Processor -- start 2024-04-25T21:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 -- start start 2024-04-25T21:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c8b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01ada70 con 0x563fa2c68620 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01af5b0 con 0x563fa2c68f40 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8da43a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01ae140 con 0x563fa2c73800 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8d54bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8d54bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35526/0 (socket says 172.21.15.37:35526) 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.988+0000 7fb8d54bc640 1 -- 172.21.15.37:0/57193564 learned_addr learned my addr 172.21.15.37:0/57193564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d5cbd640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c8b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 -- 172.21.15.37:0/57193564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 msgr2=0x563fa2c8b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c8b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 -- 172.21.15.37:0/57193564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 msgr2=0x563fa2c63b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c63b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 -- 172.21.15.37:0/57193564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa2c68d70 con 0x563fa2c68620 2024-04-25T21:22:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c63b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009b70 tx=0x7fb8cc009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff70d5ac93f9232a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8baffd640 1 -- 172.21.15.37:0/57193564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00f440 con 0x563fa2c68620 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8baffd640 1 -- 172.21.15.37:0/57193564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8cc029e50 con 0x563fa2c68620 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.989+0000 7fb8baffd640 1 -- 172.21.15.37:0/57193564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc00fc90 con 0x563fa2c68620 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- 172.21.15.37:0/57193564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 msgr2=0x563fa2c7f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009b70 tx=0x7fb8cc009ab0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- 172.21.15.37:0/57193564 shutdown_connections 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c73800 0x563fa2c8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c68f40 0x563fa2c63b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 --2- 172.21.15.37:0/57193564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7f2f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- 172.21.15.37:0/57193564 >> 172.21.15.37:0/57193564 conn(0x563fa2c64b30 msgr2=0x563fa2c63420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- 172.21.15.37:0/57193564 shutdown_connections 2024-04-25T21:22:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- 172.21.15.37:0/57193564 wait complete. 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 -- start start 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.990+0000 7fb8da43a740 1 Processor -- start 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 0x563fa2c78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c73800 0x563fa2c792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01aeee0 con 0x563fa2c68620 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01ae810 con 0x563fa2c73800 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01aba70 con 0x563fa2c68f40 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 10 client.0 _ll_register_callbacks cb 0x7fb8c009bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35536/0 (socket says 172.21.15.37:35536) 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1355777255 learned_addr learned my addr 172.21.15.37:0/1355777255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 0x563fa2c78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 msgr2=0x563fa2c78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 0x563fa2c78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1355777255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c73800 msgr2=0x563fa2c792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c73800 0x563fa2c792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 -- 172.21.15.37:0/1355777255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa2c79810 con 0x563fa2c68620 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 0x563fa2c78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8d54bc640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0117f0 tx=0x7fb8cc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368cb0fdc33ddead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc036060 con 0x563fa2c68620 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fb8cc00f910 con 0x563fa2c68620 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc029e50 con 0x563fa2c68620 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc00fb10 con 0x563fa2c68620 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 10 client.17013 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:18.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.991+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563fa2ce99e0 con 0x563fa2c68620 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8cc029e50 con 0x563fa2c68620 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb8cc029460 con 0x563fa2c68620 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 10 client.17013 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 20 client.17013 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2ce9f30 0x563fa2cec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 4 client.17013 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fa2c70cd0 con 0x563fa2ce9f30 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.992+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2ce9f30 0x563fa2cec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.993+0000 7fb8b8ff9640 10 client.17013 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:18.993+0000 7fb8d4cbb640 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2ce9f30 0x563fa2cec350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c7264a7e298a0d server_cookie=13b25784f456b14d in_seq=0 out_seq=0 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8b8ff9640 1 -- 172.21.15.37:0/1355777255 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fa2c70cd0 con 0x563fa2ce9f30 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8b8ff9640 10 client.17013 handle_command_reply: tid=0 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 client.17013 shutdown 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 msgr2=0x563fa2c7b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0117f0 tx=0x7fb8cc0095b0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 shutdown_connections 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563fa2ce9f30 0x563fa2cec350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x563fa2c73800 0x563fa2c792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563fa2c68f40 0x563fa2c78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 --2- 172.21.15.37:0/1355777255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fa2c68620 0x563fa2c7b860 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8b8ff9640 0 client.17013 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 >> 172.21.15.37:0/1355777255 conn(0x563fa2c64b30 msgr2=0x563fa2c63cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 shutdown_connections 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 1 -- 172.21.15.37:0/1355777255 wait complete. 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.004+0000 7fb8da43a740 20 client.17013 trim_cache size 0 max 16384 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:22:19.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 msgr2=0x7fb8bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 0x7fb8bc0a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8006010 tx=0x7fb8c8009a20 comp rx=0 tx=0).stop 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 msgr2=0x7fb8d0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d740 tx=0x7fb8c000dc00 comp rx=0 tx=0).stop 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 shutdown_connections 2024-04-25T21:22:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb8bc0a3dc0 0x7fb8bc0a6280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8d019cf30 0x7fb8d019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8d01072a0 0x7fb8d0198bb0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 --2- 172.21.15.37:0/314341042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8d01021d0 0x7fb8d0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 >> 172.21.15.37:0/314341042 conn(0x7fb8d00793e0 msgr2=0x7fb8d00ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 shutdown_connections 2024-04-25T21:22:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:19.005+0000 7fb8d7720640 1 -- 172.21.15.37:0/314341042 wait complete. 2024-04-25T21:22:19.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:19 smithi177 ceph-mon[33043]: pgmap v434: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:19.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:19 smithi098 ceph-mon[32569]: pgmap v434: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:19 smithi037 ceph-mon[29195]: pgmap v434: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.5 KiB/s wr, 13 op/s 2024-04-25T21:22:21.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:21 smithi177 ceph-mon[33043]: pgmap v435: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:21.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:21 smithi098 ceph-mon[32569]: pgmap v435: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:21 smithi037 ceph-mon[29195]: pgmap v435: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:23.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:23 smithi177 ceph-mon[33043]: pgmap v436: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:23.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:23 smithi098 ceph-mon[32569]: pgmap v436: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:23 smithi037 ceph-mon[29195]: pgmap v436: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8.8 KiB/s wr, 13 op/s 2024-04-25T21:22:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:25 smithi177 ceph-mon[33043]: pgmap v437: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-25T21:22:25.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:25 smithi098 ceph-mon[32569]: pgmap v437: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-25T21:22:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:25 smithi037 ceph-mon[29195]: pgmap v437: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.9 KiB/s wr, 13 op/s 2024-04-25T21:22:27.786 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:27 smithi037 ceph-mon[29195]: pgmap v438: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-25T21:22:27.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:27 smithi177 ceph-mon[33043]: pgmap v438: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-25T21:22:27.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:27 smithi098 ceph-mon[32569]: pgmap v438: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-25T21:22:28.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:22:28.907 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:22:29.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:29 smithi037 ceph-mon[29195]: pgmap v439: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-04-25T21:22:29.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:29 smithi177 ceph-mon[33043]: pgmap v439: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-04-25T21:22:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:29 smithi098 ceph-mon[32569]: pgmap v439: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.6 KiB/s wr, 10 op/s 2024-04-25T21:22:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/2549908673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4101940 msgr2=0x7fd1c4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/2549908673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4101940 0x7fd1c4101dc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009990 tx=0x7fd1b8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/2549908673 shutdown_connections 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/2549908673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 0x7fd1c410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/2549908673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4107990 0x7fd1c4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/2549908673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4101940 0x7fd1c4101dc0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/2549908673 >> 172.21.15.37:0/2549908673 conn(0x7fd1c40788f0 msgr2=0x7fd1c4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/2549908673 shutdown_connections 2024-04-25T21:22:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.605+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/2549908673 wait complete. 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.606+0000 7fd1ca9cf700 1 Processor -- start 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.606+0000 7fd1ca9cf700 1 -- start start 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.606+0000 7fd1ca9cf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4101940 0x7fd1c4199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.606+0000 7fd1ca9cf700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 0x7fd1c419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.606+0000 7fd1ca9cf700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1ca9cf700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111b00 con 0x7fd1c4107990 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1ca9cf700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111980 con 0x7fd1c4102300 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1ca9cf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c4111c80 con 0x7fd1c4101940 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57894/0 (socket says 172.21.15.37:57894) 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 -- 172.21.15.37:0/302445955 learned_addr learned my addr 172.21.15.37:0/302445955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c37fe700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 0x7fd1c419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:30.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c3fff700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4101940 0x7fd1c4199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 -- 172.21.15.37:0/302445955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4101940 msgr2=0x7fd1c4199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4101940 0x7fd1c4199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 -- 172.21.15.37:0/302445955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 msgr2=0x7fd1c419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 0x7fd1c419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 -- 172.21.15.37:0/302445955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c41a5540 con 0x7fd1c4107990 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c8f6c700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc007ee0 tx=0x7fd1bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00dbd0 con 0x7fd1c4107990 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc00dd70 con 0x7fd1c4107990 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c41a5770 con 0x7fd1c4107990 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc005990 con 0x7fd1c4107990 2024-04-25T21:22:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.607+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c41a5cd0 con 0x7fd1c4107990 2024-04-25T21:22:30.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.609+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c4109eb0 con 0x7fd1c4107990 2024-04-25T21:22:30.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.610+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1bc01a020 con 0x7fd1c4107990 2024-04-25T21:22:30.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.610+0000 7fd1c17fa700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 0x7fd1ac0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:30.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.610+0000 7fd1c3fff700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 0x7fd1ac0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:30.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.610+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fd1bc0cff30 con 0x7fd1c4107990 2024-04-25T21:22:30.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.611+0000 7fd1c3fff700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 0x7fd1ac0a62f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b4005d90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:30.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.615+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1bc010070 con 0x7fd1c4107990 2024-04-25T21:22:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.869+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1c4109890 con 0x7fd1c4107990 2024-04-25T21:22:30.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.869+0000 7fd1c17fa700 1 -- 172.21.15.37:0/302445955 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fd1bc09c5d0 con 0x7fd1c4107990 2024-04-25T21:22:30.869 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:22:30.870 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:22:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.871+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 msgr2=0x7fd1ac0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.871+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 0x7fd1ac0a62f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005fd0 tx=0x7fd1b4005d90 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 msgr2=0x7fd1c41a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc007ee0 tx=0x7fd1bc00d3b0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 shutdown_connections 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd1ac0a3e30 0x7fd1ac0a62f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1c4107990 0x7fd1c41a5000 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd1c4101940 0x7fd1c4199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 --2- 172.21.15.37:0/302445955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1c4102300 0x7fd1c419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 >> 172.21.15.37:0/302445955 conn(0x7fd1c40788f0 msgr2=0x7fd1c40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 shutdown_connections 2024-04-25T21:22:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:30.872+0000 7fd1ca9cf700 1 -- 172.21.15.37:0/302445955 wait complete. 2024-04-25T21:22:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:22:31.570 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 9009a9a9 --release --await' 2024-04-25T21:22:31.570 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 9009a9a9 --release --await 2024-04-25T21:22:31.825 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:22:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:31 smithi177 ceph-mon[33043]: pgmap v440: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-04-25T21:22:31.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:31 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/302445955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:22:31.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:31 smithi098 ceph-mon[32569]: pgmap v440: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-04-25T21:22:31.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:31 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/302445955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:22:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:31 smithi037 ceph-mon[29195]: pgmap v440: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2024-04-25T21:22:31.948 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:31 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/302445955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:22:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 -- 172.21.15.37:0/2017610425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa0107290 msgr2=0x7fdfa010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 --2- 172.21.15.37:0/2017610425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa0107290 0x7fdfa010ee60 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009ca0 tx=0x7fdf9c02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 -- 172.21.15.37:0/2017610425 shutdown_connections 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 --2- 172.21.15.37:0/2017610425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa0107290 0x7fdfa010ee60 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 --2- 172.21.15.37:0/2017610425 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa0102b70 0x7fdfa0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 --2- 172.21.15.37:0/2017610425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa01021c0 0x7fdfa01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 -- 172.21.15.37:0/2017610425 >> 172.21.15.37:0/2017610425 conn(0x7fdfa00fda20 msgr2=0x7fdfa00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 -- 172.21.15.37:0/2017610425 shutdown_connections 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 -- 172.21.15.37:0/2017610425 wait complete. 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.095+0000 7fdfa741e640 1 Processor -- start 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 -- start start 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa01021c0 0x7fdfa0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0107290 0x7fdfa0198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01121e0 con 0x7fdfa019ced0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0112060 con 0x7fdfa0107290 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa741e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0112360 con 0x7fdfa01021c0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57906/0 (socket says 172.21.15.37:57906) 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 -- 172.21.15.37:0/1433892901 learned_addr learned my addr 172.21.15.37:0/1433892901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa01021c0 0x7fdfa0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0107290 0x7fdfa0198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 -- 172.21.15.37:0/1433892901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa01021c0 msgr2=0x7fdfa0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa01021c0 0x7fdfa0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 -- 172.21.15.37:0/1433892901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0107290 msgr2=0x7fdfa0198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0107290 0x7fdfa0198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 -- 172.21.15.37:0/1433892901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa019d9c0 con 0x7fdfa019ced0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdfa59bb640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005cc0 tx=0x7fdf9c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.096+0000 7fdf967fc640 1 -- 172.21.15.37:0/1433892901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c003bf0 con 0x7fdfa019ced0 2024-04-25T21:22:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.097+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa019dc50 con 0x7fdfa019ced0 2024-04-25T21:22:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.097+0000 7fdf967fc640 1 -- 172.21.15.37:0/1433892901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9c003d90 con 0x7fdfa019ced0 2024-04-25T21:22:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.097+0000 7fdf967fc640 1 -- 172.21.15.37:0/1433892901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c02fb20 con 0x7fdfa019ced0 2024-04-25T21:22:32.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.097+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa01a5450 con 0x7fdfa019ced0 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdf967fc640 1 -- 172.21.15.37:0/1433892901 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf9c004580 con 0x7fdfa019ced0 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 Processor -- start 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 -- start start 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e5093f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 0x55e5094042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdf967fc640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 0x7fdf8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e50935e970 con 0x55e509404800 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e5092fb4a0 con 0x55e5093f9620 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.099+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e50935a930 con 0x55e5093f9f40 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa49b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa59bb640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 0x7fdf8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa49b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57920/0 (socket says 172.21.15.37:57920) 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa49b9640 1 -- 172.21.15.37:0/2433165948 learned_addr learned my addr 172.21.15.37:0/2433165948 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 0x55e5094042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdf967fc640 1 -- 172.21.15.37:0/1433892901 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fdf9c0f1c90 con 0x7fdfa019ced0 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e5093f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 -- 172.21.15.37:0/2433165948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 msgr2=0x55e5093f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e5093f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 -- 172.21.15.37:0/2433165948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 msgr2=0x55e509406bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 -- 172.21.15.37:0/2433165948 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e509071210 con 0x55e5093f9f40 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa59bb640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 0x7fdf8c0a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ef9e0 tx=0x7fdf9c02ee80 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.100+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 0x55e5094042c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800abf0 tx=0x7fdf9800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25cca1a823e05ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdf8b7fe640 1 -- 172.21.15.37:0/2433165948 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98005220 con 0x55e5093f9f40 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdf8b7fe640 1 -- 172.21.15.37:0/2433165948 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf980047a0 con 0x55e5093f9f40 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdf8b7fe640 1 -- 172.21.15.37:0/2433165948 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf980186a0 con 0x55e5093f9f40 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 -- 172.21.15.37:0/2433165948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 msgr2=0x55e5094042c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 0x55e5094042c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800abf0 tx=0x7fdf9800f790 comp rx=0 tx=0).stop 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 -- 172.21.15.37:0/2433165948 shutdown_connections 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509406bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9f40 0x55e5094042c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/2433165948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e5093f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 -- 172.21.15.37:0/2433165948 >> 172.21.15.37:0/2433165948 conn(0x55e5093f5b30 msgr2=0x55e5093f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 -- 172.21.15.37:0/2433165948 shutdown_connections 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.101+0000 7fdfaa20d740 1 -- 172.21.15.37:0/2433165948 wait complete. 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 -- start start 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 Processor -- start 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e509404080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5094018b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e509407430 con 0x55e5093f9f40 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55e5092f90b0 con 0x55e5093f9620 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55e5094075b0 con 0x55e509404800 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfaa20d740 10 client.0 _ll_register_callbacks cb 0x55e5092e6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfa51ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfa51ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57932/0 (socket says 172.21.15.37:57932) 2024-04-25T21:22:32.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfa51ba640 1 -- 172.21.15.37:0/1628745299 learned_addr learned my addr 172.21.15.37:0/1628745299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5094018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.102+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e509404080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 -- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 msgr2=0x55e5094018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5094018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 -- 172.21.15.37:0/1628745299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 msgr2=0x55e509404080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e509404080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 -- 172.21.15.37:0/1628745299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e50913d620 con 0x55e5093f9f40 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdf98010010 tx=0x7fdf980051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c4ff932dbcc25c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98027070 con 0x55e5093f9f40 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fdf98005580 con 0x55e5093f9f40 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf980047a0 con 0x55e5093f9f40 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf98033050 con 0x55e5093f9f40 2024-04-25T21:22:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfaa20d740 10 client.17031 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e5094132c0 con 0x55e5093f9f40 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf980047a0 con 0x55e5093f9f40 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.103+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf98023430 con 0x55e5093f9f40 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 10 client.17031 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 20 client.17031 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e5094146d0 0x55e509416ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 4 client.17031 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e50936c390 con 0x55e5094146d0 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e5094146d0 0x55e509416ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e5094146d0 0x55e509416ac0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d225f2fe2a66998 server_cookie=3e2cc54ae1e72bc1 in_seq=0 out_seq=0 2024-04-25T21:22:32.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.104+0000 7fdf88ff9640 10 client.17031 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdf88ff9640 1 -- 172.21.15.37:0/1628745299 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e50936c390 con 0x55e5094146d0 2024-04-25T21:22:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdf88ff9640 10 client.17031 handle_command_reply: tid=0 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 client.17031 shutdown 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 msgr2=0x55e509401370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fdf98010010 tx=0x7fdf980051d0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 shutdown_connections 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e5094146d0 0x55e509416ac0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5094018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e509401370 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdf88ff9640 0 client.17031 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1628745299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e509404080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 >> 172.21.15.37:0/1628745299 conn(0x55e5093f5b30 msgr2=0x55e5093fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 shutdown_connections 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1628745299 wait complete. 2024-04-25T21:22:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.105+0000 7fdfaa20d740 20 client.17031 trim_cache size 0 max 16384 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 Processor -- start 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 -- start start 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9620 0x55e5094102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 0x55e5093fc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01ad9d0 con 0x55e509404800 2024-04-25T21:22:32.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01af510 con 0x55e5093f9f40 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01ae0a0 con 0x55e5093f9620 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfa49b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfa49b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57936/0 (socket says 172.21.15.37:57936) 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfa49b9640 1 -- 172.21.15.37:0/1913212074 learned_addr learned my addr 172.21.15.37:0/1913212074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9620 0x55e5094102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.121+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 0x55e5093fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 -- 172.21.15.37:0/1913212074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9620 msgr2=0x55e5094102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9620 0x55e5094102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 -- 172.21.15.37:0/1913212074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 msgr2=0x55e5093fc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 0x55e5093fc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 -- 172.21.15.37:0/1913212074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5093fa6b0 con 0x55e509404800 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 0x55e5093fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058c0 tx=0x7fdf90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfa995032e7a2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdf8affd640 1 -- 172.21.15.37:0/1913212074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9002b070 con 0x55e509404800 2024-04-25T21:22:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdf8affd640 1 -- 172.21.15.37:0/1913212074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf90012e00 con 0x55e509404800 2024-04-25T21:22:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.122+0000 7fdf8affd640 1 -- 172.21.15.37:0/1913212074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900039f0 con 0x55e509404800 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1913212074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 msgr2=0x55e509403bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058c0 tx=0x7fdf90004730 comp rx=0 tx=0).stop 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1913212074 shutdown_connections 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e509404800 0x55e509403bb0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9f40 0x55e5093fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/1913212074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e5093f9620 0x55e5094102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1913212074 >> 172.21.15.37:0/1913212074 conn(0x55e5093f5b30 msgr2=0x55e5093fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1913212074 shutdown_connections 2024-04-25T21:22:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- 172.21.15.37:0/1913212074 wait complete. 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 -- start start 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.123+0000 7fdfaa20d740 1 Processor -- start 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e50940c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5093fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01aee40 con 0x55e5093f9f40 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01ae770 con 0x55e5093f9620 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa01ab9d0 con 0x55e509404800 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 10 client.0 _ll_register_callbacks cb 0x7fdf9800aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57938/0 (socket says 172.21.15.37:57938) 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 -- 172.21.15.37:0/195393334 learned_addr learned my addr 172.21.15.37:0/195393334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e50940c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 -- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 msgr2=0x55e5093fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5093fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 -- 172.21.15.37:0/195393334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 msgr2=0x55e50940c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e50940c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 -- 172.21.15.37:0/195393334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5093fe660 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfa51ba640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004ca0 tx=0x7fdf980052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858cd4d22b9f4f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdf88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98027040 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fdf98005580 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.124+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf90012e00 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf98023430 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 10 client.17037 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e50947a680 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf90012e00 con 0x55e5093f9f40 2024-04-25T21:22:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdf9802c8f0 con 0x55e5093f9f40 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 10 client.17037 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 20 client.17037 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e50947ad10 0x55e50947d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 4 client.17037 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e509401c40 con 0x55e50947ad10 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.125+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e50947ad10 0x55e50947d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.126+0000 7fdf88ff9640 10 client.17037 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:32.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.126+0000 7fdfa49b9640 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e50947ad10 0x55e50947d130 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c54bbc30c957b44 server_cookie=80d11958bc6becc2 in_seq=0 out_seq=0 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.133+0000 7fdf88ff9640 1 -- 172.21.15.37:0/195393334 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e509401c40 con 0x55e50947ad10 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.133+0000 7fdf88ff9640 10 client.17037 handle_command_reply: tid=0 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.133+0000 7fdfaa20d740 1 client.17037 shutdown 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.133+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 msgr2=0x55e5093fdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.133+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fdf98004ca0 tx=0x7fdf980052c0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 shutdown_connections 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55e50947ad10 0x55e50947d130 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e509404800 0x55e5093fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5093f9f40 0x55e5093fdbe0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdf88ff9640 0 client.17037 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 --2- 172.21.15.37:0/195393334 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55e5093f9620 0x55e50940c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 >> 172.21.15.37:0/195393334 conn(0x55e5093f5b30 msgr2=0x55e5093fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 shutdown_connections 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 1 -- 172.21.15.37:0/195393334 wait complete. 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfaa20d740 20 client.17037 trim_cache size 0 max 16384 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:22:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 msgr2=0x7fdf8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 0x7fdf8c0a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0ef9e0 tx=0x7fdf9c02ee80 comp rx=0 tx=0).stop 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 msgr2=0x7fdfa019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.134+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c005cc0 tx=0x7fdf9c02f380 comp rx=0 tx=0).stop 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 shutdown_connections 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfa019ced0 0x7fdfa019d380 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0107290 0x7fdfa0198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdf8c0a3dc0 0x7fdf8c0a6280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 --2- 172.21.15.37:0/1433892901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdfa01021c0 0x7fdfa0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 >> 172.21.15.37:0/1433892901 conn(0x7fdfa00fda20 msgr2=0x7fdfa00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 shutdown_connections 2024-04-25T21:22:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:32.135+0000 7fdfa741e640 1 -- 172.21.15.37:0/1433892901 wait complete. 2024-04-25T21:22:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 -- 172.21.15.37:0/2342096078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190107990 msgr2=0x7fc190107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 --2- 172.21.15.37:0/2342096078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190107990 0x7fc190107d70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc184009990 tx=0x7fc184009ca0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 -- 172.21.15.37:0/2342096078 shutdown_connections 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 --2- 172.21.15.37:0/2342096078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190102300 0x7fc19010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 --2- 172.21.15.37:0/2342096078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190107990 0x7fc190107d70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 --2- 172.21.15.37:0/2342096078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 0x7fc190101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 -- 172.21.15.37:0/2342096078 >> 172.21.15.37:0/2342096078 conn(0x7fc1900788f0 msgr2=0x7fc190078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 -- 172.21.15.37:0/2342096078 shutdown_connections 2024-04-25T21:22:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.511+0000 7fc196aab700 1 -- 172.21.15.37:0/2342096078 wait complete. 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.512+0000 7fc196aab700 1 Processor -- start 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.512+0000 7fc196aab700 1 -- start start 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 0x7fc190198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 0x7fc19019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111ae0 con 0x7fc190102300 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111960 con 0x7fc190101940 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc196aab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc190111c60 con 0x7fc190107990 2024-04-25T21:22:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57962/0 (socket says 172.21.15.37:57962) 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 -- 172.21.15.37:0/2925082564 learned_addr learned my addr 172.21.15.37:0/2925082564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc194847700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 0x7fc190198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc195048700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 0x7fc19019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 -- 172.21.15.37:0/2925082564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 msgr2=0x7fc19019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 0x7fc19019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 -- 172.21.15.37:0/2925082564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 msgr2=0x7fc190198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 0x7fc190198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 -- 172.21.15.37:0/2925082564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019d020 con 0x7fc190102300 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc195048700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 0x7fc19019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.513+0000 7fc18ffff700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007b50 tx=0x7fc17c007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.514+0000 7fc18dffb700 1 -- 172.21.15.37:0/2925082564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c01a070 con 0x7fc190102300 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.514+0000 7fc18dffb700 1 -- 172.21.15.37:0/2925082564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc17c00ea70 con 0x7fc190102300 2024-04-25T21:22:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.514+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019d310 con 0x7fc190102300 2024-04-25T21:22:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.514+0000 7fc18dffb700 1 -- 172.21.15.37:0/2925082564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00aa30 con 0x7fc190102300 2024-04-25T21:22:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.514+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19006a720 con 0x7fc190102300 2024-04-25T21:22:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.516+0000 7fc18dffb700 1 -- 172.21.15.37:0/2925082564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc17c00ac10 con 0x7fc190102300 2024-04-25T21:22:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.517+0000 7fc18dffb700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 0x7fc1800a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.517+0000 7fc194847700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 0x7fc1800a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.517+0000 7fc18dffb700 1 -- 172.21.15.37:0/2925082564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fc17c0d16a0 con 0x7fc190102300 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.517+0000 7fc1a1334b80 1 Processor -- start 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 -- start start 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 0x7fc1800a62f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc184006010 tx=0x7fc18401a040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5614da920d90 con 0x5614da9d8e70 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5614da8c7a30 con 0x5614da9cd650 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614da91cd50 con 0x5614da9cdf70 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc195048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:56064/0 (socket says 172.21.15.37:56064) 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc195048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57970/0 (socket says 172.21.15.37:57970) 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 -- 172.21.15.37:0/1921820837 learned_addr learned my addr 172.21.15.37:0/1921820837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc18ffff700 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 -- 172.21.15.37:0/1921820837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 msgr2=0x5614da9d8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 -- 172.21.15.37:0/1921820837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 msgr2=0x5614da9db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc194847700 1 -- 172.21.15.37:0/1921820837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614da88b840 con 0x5614da9cd650 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.518+0000 7fc195048700 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.519+0000 7fc194847700 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc18400bab0 tx=0x7fc18401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee960f99c17c0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.519+0000 7fc17affd700 1 -- 172.21.15.37:0/1921820837 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400f8f0 con 0x5614da9cd650 2024-04-25T21:22:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.519+0000 7fc17affd700 1 -- 172.21.15.37:0/1921820837 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184032070 con 0x5614da9cd650 2024-04-25T21:22:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.519+0000 7fc17affd700 1 -- 172.21.15.37:0/1921820837 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184023e40 con 0x5614da9cd650 2024-04-25T21:22:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- 172.21.15.37:0/1921820837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 msgr2=0x5614da9cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc18400bab0 tx=0x7fc18401e000 comp rx=0 tx=0).stop 2024-04-25T21:22:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- 172.21.15.37:0/1921820837 shutdown_connections 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9cda30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/1921820837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- 172.21.15.37:0/1921820837 >> 172.21.15.37:0/1921820837 conn(0x5614da8fa990 msgr2=0x5614da9c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- 172.21.15.37:0/1921820837 shutdown_connections 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- 172.21.15.37:0/1921820837 wait complete. 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 -- start start 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.520+0000 7fc1a1334b80 1 Processor -- start 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5614da9dfcb0 con 0x5614da9d8e70 2024-04-25T21:22:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5614da8da6b0 con 0x5614da9cd650 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5614da9dfe30 con 0x5614da9cdf70 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc1a1334b80 10 client.0 _ll_register_callbacks cb 0x5614da8b9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc194847700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc194847700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.37:56076/0 (socket says 172.21.15.37:56076) 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc194847700 1 -- 172.21.15.37:0/2547903905 learned_addr learned my addr 172.21.15.37:0/2547903905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc18ffff700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 -- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 msgr2=0x5614da9d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 -- 172.21.15.37:0/2547903905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 msgr2=0x5614da9d85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.521+0000 7fc195048700 1 -- 172.21.15.37:0/2547903905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614da86e9d0 con 0x5614da9d8e70 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.522+0000 7fc195048700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d5e20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc1880078e0 tx=0x7fc188007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e395d1d3f3b1dedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.522+0000 7fc16ffff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.523+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188019030 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc184032070 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fc188012040 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc188023050 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 10 client.17049 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614da9ea610 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184032070 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc188011450 con 0x5614da9d8e70 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 10 client.17049 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 10 client.17049 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 20 client.17049 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614da9eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 4 client.17049 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.524+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614da92e580 con 0x5614da9ecba0 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.525+0000 7fc194847700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614da9eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.525+0000 7fc16ffff700 10 client.17049 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.525+0000 7fc194847700 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614da9eefb0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29fa350b2bb215d server_cookie=f061fae01fb6ed75 in_seq=0 out_seq=0 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc16ffff700 1 -- 172.21.15.37:0/2547903905 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614da92e580 con 0x5614da9ecba0 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc16ffff700 10 client.17049 handle_command_reply: tid=0 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 client.17049 shutdown 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 msgr2=0x5614da9d5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d5e20 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc1880078e0 tx=0x7fc188007bf0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 shutdown_connections 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614da9eefb0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc16ffff700 0 client.17049 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d5e20 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/2547903905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 >> 172.21.15.37:0/2547903905 conn(0x5614da8fa990 msgr2=0x5614da9cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 shutdown_connections 2024-04-25T21:22:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 1 -- 172.21.15.37:0/2547903905 wait complete. 2024-04-25T21:22:33.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.526+0000 7fc1a1334b80 20 client.17049 trim_cache size 0 max 16384 2024-04-25T21:22:33.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 Processor -- start 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 -- start start 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ab080 con 0x5614da9d8e70 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901acbc0 con 0x5614da9cd650 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ab750 con 0x5614da9cdf70 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57992/0 (socket says 172.21.15.37:57992) 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 -- 172.21.15.37:0/3324943970 learned_addr learned my addr 172.21.15.37:0/3324943970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc194847700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc18ffff700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 -- 172.21.15.37:0/3324943970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 msgr2=0x5614da9e8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9e8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 -- 172.21.15.37:0/3324943970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 msgr2=0x5614da9d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 -- 172.21.15.37:0/3324943970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614da9d3e90 con 0x5614da9d8e70 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc18ffff700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.546+0000 7fc195048700 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc188007630 tx=0x7fc1880096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ad1ed618d3bd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc17a7fc700 1 -- 172.21.15.37:0/3324943970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188019020 con 0x5614da9d8e70 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc17a7fc700 1 -- 172.21.15.37:0/3324943970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc188005c60 con 0x5614da9d8e70 2024-04-25T21:22:33.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc17a7fc700 1 -- 172.21.15.37:0/3324943970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188005830 con 0x5614da9d8e70 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 -- 172.21.15.37:0/3324943970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 msgr2=0x5614da9d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc188007630 tx=0x7fc1880096e0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 -- 172.21.15.37:0/3324943970 shutdown_connections 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9d8e70 0x5614da9d2bc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/3324943970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9cdf70 0x5614da9e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 -- 172.21.15.37:0/3324943970 >> 172.21.15.37:0/3324943970 conn(0x5614da8fa990 msgr2=0x5614da9d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 -- 172.21.15.37:0/3324943970 shutdown_connections 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.547+0000 7fc1a1334b80 1 -- 172.21.15.37:0/3324943970 wait complete. 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 -- start start 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 Processor -- start 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9d8e70 0x5614da9e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ac4f0 con 0x5614da9cdf70 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901abe20 con 0x5614da9cd650 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901a9040 con 0x5614da9d8e70 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc1a1334b80 10 client.0 _ll_register_callbacks cb 0x5614da8b9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:33.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc18ffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58008/0 (socket says 172.21.15.37:58008) 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc18ffff700 1 -- 172.21.15.37:0/951433271 learned_addr learned my addr 172.21.15.37:0/951433271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.548+0000 7fc194847700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc195048700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9d8e70 0x5614da9e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 -- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9d8e70 msgr2=0x5614da9e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9d8e70 0x5614da9e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 -- 172.21.15.37:0/951433271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 msgr2=0x5614da9e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 -- 172.21.15.37:0/951433271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614da9e2050 con 0x5614da9cdf70 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc18ffff700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc17c09d5d0 tx=0x7fc17c09d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d5c04887326b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc16ffff700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0a62b0 con 0x5614da9cdf70 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fc17c09dab0 con 0x5614da9cdf70 2024-04-25T21:22:33.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc188005c60 con 0x5614da9cdf70 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc17c0a5e20 con 0x5614da9cdf70 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc1a1334b80 10 client.17055 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614daa53e90 con 0x5614da9cdf70 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.549+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc188005c60 con 0x5614da9cdf70 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc17c014070 con 0x5614da9cdf70 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 10 client.17055 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 10 client.17055 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 20 client.17055 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614daa565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 4 client.17055 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9009a9a9", "release": true, "await": true}] 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9009a9a9", "release": true, "await": true}) -- 0x5614da928620 con 0x5614da9ecba0 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc194847700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614daa565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc194847700 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614daa565e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b71e5bf577e17b8 server_cookie=f750cb5f557077ed in_seq=0 out_seq=0 2024-04-25T21:22:33.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.550+0000 7fc16ffff700 10 client.17055 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:33.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc16ffff700 1 -- 172.21.15.37:0/951433271 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5614da928620 con 0x5614da9ecba0 2024-04-25T21:22:33.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc16ffff700 10 client.17055 handle_command_reply: tid=0 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 client.17055 shutdown 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 msgr2=0x5614da9e15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc17c09d5d0 tx=0x7fc17c09d070 comp rx=0 tx=0).stop 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 shutdown_connections 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5614da9ecba0 0x5614daa565e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5614da9d8e70 0x5614da9e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5614da9cd650 0x5614da9e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc16ffff700 0 client.17055 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 --2- 172.21.15.37:0/951433271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5614da9cdf70 0x5614da9e15a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 >> 172.21.15.37:0/951433271 conn(0x5614da8fa990 msgr2=0x5614da9e8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 shutdown_connections 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 1 -- 172.21.15.37:0/951433271 wait complete. 2024-04-25T21:22:33.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.554+0000 7fc1a1334b80 20 client.17055 trim_cache size 0 max 16384 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 40, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "9009a9a9": { 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 40, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:22:33.557 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 86.5, 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:33.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 msgr2=0x7fc1800a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 0x7fc1800a62f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc184006010 tx=0x7fc18401a040 comp rx=0 tx=0).stop 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 msgr2=0x7fc190198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007b50 tx=0x7fc17c007f10 comp rx=0 tx=0).stop 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 shutdown_connections 2024-04-25T21:22:33.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc1800a3e30 0x7fc1800a62f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc190107990 0x7fc19019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc190101940 0x7fc190198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 --2- 172.21.15.37:0/2925082564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc190102300 0x7fc190198550 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 >> 172.21.15.37:0/2925082564 conn(0x7fc1900788f0 msgr2=0x7fc1901007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 shutdown_connections 2024-04-25T21:22:33.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:22:33.555+0000 7fc196aab700 1 -- 172.21.15.37:0/2925082564 wait complete. 2024-04-25T21:22:33.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:33 smithi177 ceph-mon[33043]: pgmap v441: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8 op/s 2024-04-25T21:22:33.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:33 smithi098 ceph-mon[32569]: pgmap v441: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8 op/s 2024-04-25T21:22:33.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:33 smithi037 ceph-mon[29195]: pgmap v441: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8 op/s 2024-04-25T21:22:34.285 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9009a9a9', seconds elapsed: 37.9 2024-04-25T21:22:35.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:35 smithi177 ceph-mon[33043]: pgmap v442: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T21:22:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:35 smithi098 ceph-mon[32569]: pgmap v442: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T21:22:35.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:35 smithi037 ceph-mon[29195]: pgmap v442: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-25T21:22:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:37 smithi177 ceph-mon[33043]: pgmap v443: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:37 smithi098 ceph-mon[32569]: pgmap v443: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:37.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:37 smithi037 ceph-mon[29195]: pgmap v443: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:39 smithi177 ceph-mon[33043]: pgmap v444: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:39.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:39 smithi098 ceph-mon[32569]: pgmap v444: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:39 smithi037 ceph-mon[29195]: pgmap v444: 97 pgs: 97 active+clean; 2.7 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.9 KiB/s wr, 10 op/s 2024-04-25T21:22:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:41 smithi177 ceph-mon[33043]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T21:22:41.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:41 smithi098 ceph-mon[32569]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T21:22:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:41 smithi037 ceph-mon[29195]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T21:22:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:43 smithi177 ceph-mon[33043]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-25T21:22:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:43 smithi098 ceph-mon[32569]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-25T21:22:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:43 smithi037 ceph-mon[29195]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-25T21:22:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 -- 172.21.15.37:0/3531694924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540742f0 msgr2=0x7fe7540746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 --2- 172.21.15.37:0/3531694924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540742f0 0x7fe7540746d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe750009900 tx=0x7fe75002a480 comp rx=0 tx=0).stop 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 -- 172.21.15.37:0/3531694924 shutdown_connections 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 --2- 172.21.15.37:0/3531694924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 0x7fe754115350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 --2- 172.21.15.37:0/3531694924 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe754074c10 0x7fe754112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 --2- 172.21.15.37:0/3531694924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7540742f0 0x7fe7540746d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 -- 172.21.15.37:0/3531694924 >> 172.21.15.37:0/3531694924 conn(0x7fe7540fda20 msgr2=0x7fe7540ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 -- 172.21.15.37:0/3531694924 shutdown_connections 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.235+0000 7fe75b89a640 1 -- 172.21.15.37:0/3531694924 wait complete. 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 Processor -- start 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 -- start start 2024-04-25T21:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7540742f0 0x7fe75419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 0x7fe7541a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541187f0 con 0x7fe754074c10 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe754118670 con 0x7fe7540742f0 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe75b89a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe754118970 con 0x7fe754112f60 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58510/0 (socket says 172.21.15.37:58510) 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 -- 172.21.15.37:0/928513408 learned_addr learned my addr 172.21.15.37:0/928513408 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe759e37640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 0x7fe7541a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe759636640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7540742f0 0x7fe75419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 -- 172.21.15.37:0/928513408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 msgr2=0x7fe7541a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 0x7fe7541a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 -- 172.21.15.37:0/928513408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7540742f0 msgr2=0x7fe75419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7540742f0 0x7fe75419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.236+0000 7fe758e35640 1 -- 172.21.15.37:0/928513408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541a3d20 con 0x7fe754074c10 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe758e35640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e9f0 tx=0x7fe74400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe74a7fc640 1 -- 172.21.15.37:0/928513408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400cd80 con 0x7fe754074c10 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe74a7fc640 1 -- 172.21.15.37:0/928513408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7440108e0 con 0x7fe754074c10 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7541a4010 con 0x7fe754074c10 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe74a7fc640 1 -- 172.21.15.37:0/928513408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744018980 con 0x7fe754074c10 2024-04-25T21:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.237+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541ab8a0 con 0x7fe754074c10 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.239+0000 7fe74a7fc640 1 -- 172.21.15.37:0/928513408 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe744010430 con 0x7fe754074c10 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 Processor -- start 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 -- start start 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe74a7fc640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 0x7fe7400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230c02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b2301a970 con 0x560b230b5620 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b22fb74a0 con 0x560b230b5f40 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560b23016930 con 0x560b230c0800 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe759e37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe759636640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 0x7fe7400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe759e37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58512/0 (socket says 172.21.15.37:58512) 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe759e37640 1 -- 172.21.15.37:0/3959810313 learned_addr learned my addr 172.21.15.37:0/3959810313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.240+0000 7fe74a7fc640 1 -- 172.21.15.37:0/928513408 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe7440d0b80 con 0x7fe754074c10 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe758e35640 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230c02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 -- 172.21.15.37:0/3959810313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 msgr2=0x560b230c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 -- 172.21.15.37:0/3959810313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 msgr2=0x560b230c02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230c02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 -- 172.21.15.37:0/3959810313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b22d2d210 con 0x560b230b5620 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759e37640 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x560b230b2cc0 tx=0x7fe74c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28864d0c7079045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe759636640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 0x7fe7400a6300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe750031440 tx=0x7fe7500095f0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe73f7fe640 1 -- 172.21.15.37:0/3959810313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003970 con 0x560b230b5620 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe73f7fe640 1 -- 172.21.15.37:0/3959810313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c00f890 con 0x560b230b5620 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.241+0000 7fe73f7fe640 1 -- 172.21.15.37:0/3959810313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004320 con 0x560b230b5620 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3959810313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 msgr2=0x560b230b5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x560b230b2cc0 tx=0x7fe74c005bd0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3959810313 shutdown_connections 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230c02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3959810313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230b5a00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3959810313 >> 172.21.15.37:0/3959810313 conn(0x560b230b1b30 msgr2=0x560b230b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3959810313 shutdown_connections 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3959810313 wait complete. 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 -- start start 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.242+0000 7fe75e5b1740 1 Processor -- start 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230bd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230bd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b230c3430 con 0x560b230b5620 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560b22fb50b0 con 0x560b230b5f40 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x560b230c35b0 con 0x560b230c0800 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe75e5b1740 10 client.0 _ll_register_callbacks cb 0x560b22fa2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58522/0 (socket says 172.21.15.37:58522) 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 -- 172.21.15.37:0/2306229574 learned_addr learned my addr 172.21.15.37:0/2306229574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe758e35640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230bd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759e37640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230bd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 -- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 msgr2=0x560b230bd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230bd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 -- 172.21.15.37:0/2306229574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 msgr2=0x560b230bd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230bd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.243+0000 7fe759636640 1 -- 172.21.15.37:0/2306229574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b22df9620 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe759636640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe750033000 tx=0x7fe750009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fa209cdaffd4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75004b070 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe750003800 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe74c00f890 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe750057050 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe75e5b1740 10 client.17067 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b230cf350 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c00f890 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe750047880 con 0x560b230b5620 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe75e5b1740 10 client.17067 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.244+0000 7fe75e5b1740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe75e5b1740 20 client.17067 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:45.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b230d0810 0x560b230d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe75e5b1740 4 client.17067 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b23028390 con 0x560b230d0810 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe758e35640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b230d0810 0x560b230d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe758e35640 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b230d0810 0x560b230d2c00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c4500337b14fcd server_cookie=6c36c0a1cc680a6d in_seq=0 out_seq=0 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.245+0000 7fe73cff9640 10 client.17067 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe73cff9640 1 -- 172.21.15.37:0/2306229574 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b23028390 con 0x560b230d0810 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe73cff9640 10 client.17067 handle_command_reply: tid=0 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 client.17067 shutdown 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 msgr2=0x560b230c0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe750033000 tx=0x7fe750009260 comp rx=0 tx=0).stop 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 shutdown_connections 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b230d0810 0x560b230d2c00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230c0800 0x560b230bd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230b5f40 0x560b230bd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe73cff9640 0 client.17067 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/2306229574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5620 0x560b230c0100 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 >> 172.21.15.37:0/2306229574 conn(0x560b230b1b30 msgr2=0x560b230b78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 shutdown_connections 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 1 -- 172.21.15.37:0/2306229574 wait complete. 2024-04-25T21:22:45.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.246+0000 7fe75e5b1740 20 client.17067 trim_cache size 0 max 16384 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.261+0000 7fe75e5b1740 1 Processor -- start 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.261+0000 7fe75e5b1740 1 -- start start 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b230d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b4160 con 0x560b230b5f40 2024-04-25T21:22:45.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b5ca0 con 0x560b230c0800 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b4830 con 0x560b230b5620 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58524/0 (socket says 172.21.15.37:58524) 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 -- 172.21.15.37:0/3735568667 learned_addr learned my addr 172.21.15.37:0/3735568667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe759e37640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b230d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe759636640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 -- 172.21.15.37:0/3735568667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 msgr2=0x560b230cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 -- 172.21.15.37:0/3735568667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 msgr2=0x560b230d8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b230d8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 -- 172.21.15.37:0/3735568667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b230b2470 con 0x560b230b5f40 2024-04-25T21:22:45.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe759636640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe758e35640 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097b0 tx=0x7fe744018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4feab8483661ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe73effd640 1 -- 172.21.15.37:0/3735568667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440af950 con 0x560b230b5f40 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe73effd640 1 -- 172.21.15.37:0/3735568667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74409cac0 con 0x560b230b5f40 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.262+0000 7fe73effd640 1 -- 172.21.15.37:0/3735568667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74409abf0 con 0x560b230b5f40 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3735568667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 msgr2=0x560b230d8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097b0 tx=0x7fe744018420 comp rx=0 tx=0).stop 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3735568667 shutdown_connections 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b230d8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230d8510 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/3735568667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3735568667 >> 172.21.15.37:0/3735568667 conn(0x560b230b1b30 msgr2=0x560b230bcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3735568667 shutdown_connections 2024-04-25T21:22:45.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- 172.21.15.37:0/3735568667 wait complete. 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 -- start start 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.263+0000 7fe75e5b1740 1 Processor -- start 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b2312ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b55d0 con 0x560b230b5f40 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b4f00 con 0x560b230c0800 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541b2160 con 0x560b230b5620 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe75e5b1740 10 client.0 _ll_register_callbacks cb 0x7fe74c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58530/0 (socket says 172.21.15.37:58530) 2024-04-25T21:22:45.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 -- 172.21.15.37:0/4173397873 learned_addr learned my addr 172.21.15.37:0/4173397873 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe759e37640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b2312ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe759636640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 -- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 msgr2=0x560b230c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 -- 172.21.15.37:0/4173397873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 msgr2=0x560b2312ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b2312ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 -- 172.21.15.37:0/4173397873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2312f340 con 0x560b230b5f40 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe759636640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe758e35640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe744099490 tx=0x7fe744000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546980ab63585142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.264+0000 7fe73cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440b5060 con 0x560b230b5f40 2024-04-25T21:22:45.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7fe744010a90 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe74409cac0 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7440a5730 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 10 client.17073 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560b23136640 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74409cac0 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe7440ce580 con 0x560b230b5f40 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 10 client.17073 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 20 client.17073 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b23136c80 0x560b231390a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 4 client.17073 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:22:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.265+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b23022200 con 0x560b23136c80 2024-04-25T21:22:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.266+0000 7fe759636640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b23136c80 0x560b231390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.266+0000 7fe759636640 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b23136c80 0x560b231390a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4948b28ee2641c1b server_cookie=8ad7d3e393d30439 in_seq=0 out_seq=0 2024-04-25T21:22:45.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.266+0000 7fe73cff9640 10 client.17073 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.277+0000 7fe73cff9640 1 -- 172.21.15.37:0/4173397873 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b23022200 con 0x560b23136c80 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.277+0000 7fe73cff9640 10 client.17073 handle_command_reply: tid=0 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.277+0000 7fe75e5b1740 1 client.17073 shutdown 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.277+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 msgr2=0x560b230c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.277+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe744099490 tx=0x7fe744000bc0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 shutdown_connections 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x560b23136c80 0x560b231390a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560b230c0800 0x560b2312ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b230b5f40 0x560b230c6010 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe73cff9640 0 client.17073 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 --2- 172.21.15.37:0/4173397873 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560b230b5620 0x560b230c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 >> 172.21.15.37:0/4173397873 conn(0x560b230b1b30 msgr2=0x560b230bcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 shutdown_connections 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 1 -- 172.21.15.37:0/4173397873 wait complete. 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75e5b1740 20 client.17073 trim_cache size 0 max 16384 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:22:45.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:22:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:22:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:22:45.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 msgr2=0x7fe7400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 0x7fe7400a6300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe750031440 tx=0x7fe7500095f0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 msgr2=0x7fe75419f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fe74400e9f0 tx=0x7fe74400eeb0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 shutdown_connections 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe7400a3e40 0x7fe7400a6300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe754112f60 0x7fe7541a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe754074c10 0x7fe75419f270 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 --2- 172.21.15.37:0/928513408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7540742f0 0x7fe75419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.278+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 >> 172.21.15.37:0/928513408 conn(0x7fe7540fda20 msgr2=0x7fe7540fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.279+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 shutdown_connections 2024-04-25T21:22:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:45.279+0000 7fe75b89a640 1 -- 172.21.15.37:0/928513408 wait complete. 2024-04-25T21:22:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:45 smithi177 ceph-mon[33043]: pgmap v447: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T21:22:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:45 smithi098 ceph-mon[32569]: pgmap v447: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T21:22:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:45 smithi037 ceph-mon[29195]: pgmap v447: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 18 op/s 2024-04-25T21:22:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:47 smithi177 ceph-mon[33043]: pgmap v448: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:47.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:47 smithi098 ceph-mon[32569]: pgmap v448: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:47 smithi037 ceph-mon[29195]: pgmap v448: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:48.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:48 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:22:48.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:48 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:22:48.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:48 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:22:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:49 smithi177 ceph-mon[33043]: pgmap v449: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:22:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:22:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:22:49.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:49 smithi098 ceph-mon[32569]: pgmap v449: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:22:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:22:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:22:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:49 smithi037 ceph-mon[29195]: pgmap v449: 97 pgs: 97 active+clean; 3.1 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-25T21:22:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:22:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:22:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:22:51.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:51 smithi177 ceph-mon[33043]: pgmap v450: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:22:51.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:51 smithi098 ceph-mon[32569]: pgmap v450: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:22:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:51 smithi037 ceph-mon[29195]: pgmap v450: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-25T21:22:53.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:53 smithi177 ceph-mon[33043]: pgmap v451: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T21:22:53.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:53 smithi098 ceph-mon[32569]: pgmap v451: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T21:22:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:53 smithi037 ceph-mon[29195]: pgmap v451: 97 pgs: 97 active+clean; 3.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-25T21:22:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:55 smithi177 ceph-mon[33043]: pgmap v452: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:22:55.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:55 smithi098 ceph-mon[32569]: pgmap v452: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:22:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:55 smithi037 ceph-mon[29195]: pgmap v452: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:22:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:57 smithi177 ceph-mon[33043]: pgmap v453: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:22:57.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:57 smithi098 ceph-mon[32569]: pgmap v453: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:22:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:57 smithi037 ceph-mon[29195]: pgmap v453: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:22:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 -- 172.21.15.37:0/2598047680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 msgr2=0x7f18e010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 --2- 172.21.15.37:0/2598047680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e010bfe0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009900 tx=0x7f18d402a510 comp rx=0 tx=0).stop 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 -- 172.21.15.37:0/2598047680 shutdown_connections 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 --2- 172.21.15.37:0/2598047680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 0x7f18e010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 --2- 172.21.15.37:0/2598047680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e010bfe0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 --2- 172.21.15.37:0/2598047680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 0x7f18e01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 -- 172.21.15.37:0/2598047680 >> 172.21.15.37:0/2598047680 conn(0x7f18e0077fd0 msgr2=0x7f18e00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 -- 172.21.15.37:0/2598047680 shutdown_connections 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.370+0000 7f18e61f7640 1 -- 172.21.15.37:0/2598047680 wait complete. 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 Processor -- start 2024-04-25T21:22:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 -- start start 2024-04-25T21:22:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 0x7f18e0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 0x7f18e019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01121b0 con 0x7f18e0103b00 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0112030 con 0x7f18e010c520 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18e61f7640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0112330 con 0x7f18e01031e0 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57038/0 (socket says 172.21.15.37:57038) 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18deffd640 1 -- 172.21.15.37:0/552321459 learned_addr learned my addr 172.21.15.37:0/552321459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18df7fe640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 0x7f18e0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.371+0000 7f18dffff640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 0x7f18e019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 -- 172.21.15.37:0/552321459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 msgr2=0x7f18e0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 0x7f18e0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 -- 172.21.15.37:0/552321459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 msgr2=0x7f18e019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 0x7f18e019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 -- 172.21.15.37:0/552321459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e019d6d0 con 0x7f18e0103b00 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18deffd640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005940 tx=0x7f18d4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18dcff9640 1 -- 172.21.15.37:0/552321459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d403f030 con 0x7f18e0103b00 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e019d960 con 0x7f18e0103b00 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18dcff9640 1 -- 172.21.15.37:0/552321459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d4003aa0 con 0x7f18e0103b00 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18dcff9640 1 -- 172.21.15.37:0/552321459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4035070 con 0x7f18e0103b00 2024-04-25T21:22:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.372+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e01a5250 con 0x7f18e0103b00 2024-04-25T21:22:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.374+0000 7f18dcff9640 1 -- 172.21.15.37:0/552321459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18d4042030 con 0x7f18e0103b00 2024-04-25T21:22:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 Processor -- start 2024-04-25T21:22:58.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 -- start start 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50807920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee508166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50819000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee5076c970 con 0x55ee50807540 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ee507094a0 con 0x55ee50807e60 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee50768930 con 0x55ee50816c10 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dcff9640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 0x7f18c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50807920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50807920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57044/0 (socket says 172.21.15.37:57044) 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dffff640 1 -- 172.21.15.37:0/1175610269 learned_addr learned my addr 172.21.15.37:0/1175610269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dffff640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 0x7f18c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee508166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18dcff9640 1 -- 172.21.15.37:0/552321459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f18d40f0cf0 con 0x7f18e0103b00 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.375+0000 7f18deffd640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50819000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 -- 172.21.15.37:0/1175610269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 msgr2=0x55ee50819000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50819000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 -- 172.21.15.37:0/1175610269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 msgr2=0x55ee50807920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50807920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 -- 172.21.15.37:0/1175610269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5047f210 con 0x55ee50807e60 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18deffd640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50819000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18dffff640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 0x7f18c80a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009fd0 tx=0x7f18d8009b90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee508166d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18d0004630 tx=0x7f18d000bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240656812819ac8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18c5ffb640 1 -- 172.21.15.37:0/1175610269 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0003bd0 con 0x55ee50807e60 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18c5ffb640 1 -- 172.21.15.37:0/1175610269 <== mon.2 v2:172.21.15.98:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d0015070 con 0x55ee50807e60 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.376+0000 7f18c5ffb640 1 -- 172.21.15.37:0/1175610269 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0004be0 con 0x55ee50807e60 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1175610269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 msgr2=0x55ee508166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee508166d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18d0004630 tx=0x7f18d000bb20 comp rx=0 tx=0).stop 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1175610269 shutdown_connections 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50819000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee508166d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1175610269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50807920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1175610269 >> 172.21.15.37:0/1175610269 conn(0x55ee50803b30 msgr2=0x55ee50803f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1175610269 shutdown_connections 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1175610269 wait complete. 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 -- start start 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.377+0000 7f18e8fe3740 1 Processor -- start 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee50813840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50813d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee50819840 con 0x55ee50807540 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ee507070b0 con 0x55ee50807e60 2024-04-25T21:22:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ee508199c0 con 0x55ee50816c10 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 10 client.0 _ll_register_callbacks cb 0x55ee506f4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57058/0 (socket says 172.21.15.37:57058) 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 -- 172.21.15.37:0/1229056523 learned_addr learned my addr 172.21.15.37:0/1229056523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18deffd640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee50813840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18dffff640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50813d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 -- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 msgr2=0x55ee50813d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50813d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 -- 172.21.15.37:0/1229056523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 msgr2=0x55ee50813840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee50813840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 -- 172.21.15.37:0/1229056523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee5054b620 con 0x55ee50807540 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18dffff640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50813d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18deffd640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee50813840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18df7fe640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f18d000bfa0 tx=0x7f18d000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de131f3b7a85edfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18beffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0028060 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d0015070 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.378+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f18d0011640 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18d0003e60 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 10 client.17085 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee508256c0 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d0015070 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18d0022070 con 0x55ee50807540 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 10 client.17085 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 20 client.17085 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee50826ac0 0x55ee50828eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 4 client.17085 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee5077a390 con 0x55ee50826ac0 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.379+0000 7f18deffd640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee50826ac0 0x55ee50828eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.380+0000 7f18beffd640 10 client.17085 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.380+0000 7f18deffd640 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee50826ac0 0x55ee50828eb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec83d8f7562722c9 server_cookie=365bedc9c141d573 in_seq=0 out_seq=0 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18beffd640 1 -- 172.21.15.37:0/1229056523 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee5077a390 con 0x55ee50826ac0 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18beffd640 10 client.17085 handle_command_reply: tid=0 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 client.17085 shutdown 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 msgr2=0x55ee50816550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f18d000bfa0 tx=0x7f18d000bfd0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 shutdown_connections 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee50826ac0 0x55ee50828eb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50813d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee50813840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18beffd640 0 client.17085 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/1229056523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50816550 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 >> 172.21.15.37:0/1229056523 conn(0x55ee50803b30 msgr2=0x55ee508099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 shutdown_connections 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 1 -- 172.21.15.37:0/1229056523 wait complete. 2024-04-25T21:22:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.381+0000 7f18e8fe3740 20 client.17085 trim_cache size 0 max 16384 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 Processor -- start 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 -- start start 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5080ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50816080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ab970 con 0x55ee50807540 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ad4b0 con 0x55ee50807e60 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ac040 con 0x55ee50816c10 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57062/0 (socket says 172.21.15.37:57062) 2024-04-25T21:22:58.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.396+0000 7f18df7fe640 1 -- 172.21.15.37:0/3223950730 learned_addr learned my addr 172.21.15.37:0/3223950730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18deffd640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5080ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18dffff640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50816080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 -- 172.21.15.37:0/3223950730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 msgr2=0x55ee50816080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50816080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 -- 172.21.15.37:0/3223950730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 msgr2=0x55ee5080ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5080ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 -- 172.21.15.37:0/3223950730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee50807c90 con 0x55ee50807540 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18deffd640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5080ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18df7fe640 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f18d0001eb0 tx=0x7f18d0019ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b06ab34398b420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18c57fa640 1 -- 172.21.15.37:0/3223950730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0028040 con 0x55ee50807540 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18c57fa640 1 -- 172.21.15.37:0/3223950730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d0015070 con 0x55ee50807540 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.397+0000 7f18c57fa640 1 -- 172.21.15.37:0/3223950730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0011ce0 con 0x55ee50807540 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- 172.21.15.37:0/3223950730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 msgr2=0x55ee50822700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f18d0001eb0 tx=0x7f18d0019ea0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- 172.21.15.37:0/3223950730 shutdown_connections 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee50816080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5080ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/3223950730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee50822700 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- 172.21.15.37:0/3223950730 >> 172.21.15.37:0/3223950730 conn(0x55ee50803b30 msgr2=0x55ee5080e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- 172.21.15.37:0/3223950730 shutdown_connections 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- 172.21.15.37:0/3223950730 wait complete. 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 -- start start 2024-04-25T21:22:58.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.398+0000 7f18e8fe3740 1 Processor -- start 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5081f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee5080fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01acde0 con 0x55ee50807540 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ac710 con 0x55ee50807e60 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01a9970 con 0x55ee50816c10 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 10 client.0 _ll_register_callbacks cb 0x7f18d00046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57068/0 (socket says 172.21.15.37:57068) 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 -- 172.21.15.37:0/81003092 learned_addr learned my addr 172.21.15.37:0/81003092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:22:58.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18dffff640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee5080fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18deffd640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5081f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 -- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 msgr2=0x55ee5080fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee5080fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 -- 172.21.15.37:0/81003092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 msgr2=0x55ee5081f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5081f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 -- 172.21.15.37:0/81003092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee50810450 con 0x55ee50807540 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18deffd640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5081f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18df7fe640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f18d0019ac0 tx=0x7f18d000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61813b0750915148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18beffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0028050 con 0x55ee50807540 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d0015070 con 0x55ee50807540 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8513+0+0 (secure 0 0 0) 0x7f18d0011850 con 0x55ee50807540 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.399+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f18d0005e50 con 0x55ee50807540 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 10 client.17091 fetch_fsmap learned FSMap version 27 2024-04-25T21:22:58.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee5088c750 con 0x55ee50807540 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d0015070 con 0x55ee50807540 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f18d0022070 con 0x55ee50807540 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 10 client.17091 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 20 client.17091 populate_metadata read hostname 'smithi037' 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee5088cdb0 0x55ee5088f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 4 client.17091 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee50774200 con 0x55ee5088cdb0 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.400+0000 7f18deffd640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee5088cdb0 0x55ee5088f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.401+0000 7f18beffd640 10 client.17091 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:58.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.401+0000 7f18deffd640 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee5088cdb0 0x55ee5088f1a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=811e311245d8b530 server_cookie=de7bd182ac040d92 in_seq=0 out_seq=0 2024-04-25T21:22:58.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18beffd640 1 -- 172.21.15.37:0/81003092 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee50774200 con 0x55ee5088cdb0 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18beffd640 10 client.17091 handle_command_reply: tid=0 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 client.17091 shutdown 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 msgr2=0x55ee5081ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f18d0019ac0 tx=0x7f18d000bf80 comp rx=0 tx=0).stop 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 shutdown_connections 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ee5088cdb0 0x55ee5088f1a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ee50816c10 0x55ee5080fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ee50807e60 0x55ee5081f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 --2- 172.21.15.37:0/81003092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee50807540 0x55ee5081ec10 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 >> 172.21.15.37:0/81003092 conn(0x55ee50803b30 msgr2=0x55ee5080ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18beffd640 0 client.17091 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 shutdown_connections 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 1 -- 172.21.15.37:0/81003092 wait complete. 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.414+0000 7f18e8fe3740 20 client.17091 trim_cache size 0 max 16384 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:22:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 msgr2=0x7f18c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 0x7f18c80a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009fd0 tx=0x7f18d8009b90 comp rx=0 tx=0).stop 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 msgr2=0x7f18e0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f18d4005940 tx=0x7f18d4004270 comp rx=0 tx=0).stop 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 shutdown_connections 2024-04-25T21:22:58.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18e010c520 0x7f18e019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e0103b00 0x7f18e0198c70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18c80a3dc0 0x7f18c80a6280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 --2- 172.21.15.37:0/552321459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f18e01031e0 0x7f18e0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 >> 172.21.15.37:0/552321459 conn(0x7f18e0077fd0 msgr2=0x7f18e00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 shutdown_connections 2024-04-25T21:22:58.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:22:58.415+0000 7f18e61f7640 1 -- 172.21.15.37:0/552321459 wait complete. 2024-04-25T21:22:59.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:22:59 smithi177 ceph-mon[33043]: pgmap v454: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:22:59.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:22:59 smithi098 ceph-mon[32569]: pgmap v454: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:22:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:22:59 smithi037 ceph-mon[29195]: pgmap v454: 97 pgs: 97 active+clean; 3.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:23:01.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:01 smithi177 ceph-mon[33043]: pgmap v455: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T21:23:01.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:01 smithi098 ceph-mon[32569]: pgmap v455: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T21:23:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:01 smithi037 ceph-mon[29195]: pgmap v455: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-25T21:23:02.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:02 smithi177 ceph-mon[33043]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T21:23:02.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:02 smithi098 ceph-mon[32569]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T21:23:02.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:02 smithi037 ceph-mon[29195]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T21:23:03.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:03 smithi177 ceph-mon[33043]: pgmap v457: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:23:03.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:03 smithi098 ceph-mon[32569]: pgmap v457: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:23:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:03 smithi037 ceph-mon[29195]: pgmap v457: 97 pgs: 97 active+clean; 3.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-25T21:23:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:05 smithi177 ceph-mon[33043]: pgmap v458: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-25T21:23:05.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:05 smithi177 ceph-mon[33043]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T21:23:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:05 smithi098 ceph-mon[32569]: pgmap v458: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-25T21:23:05.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:05 smithi098 ceph-mon[32569]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T21:23:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:05 smithi037 ceph-mon[29195]: pgmap v458: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-25T21:23:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:05 smithi037 ceph-mon[29195]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T21:23:07.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:07 smithi177 ceph-mon[33043]: pgmap v460: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:23:07.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:07 smithi098 ceph-mon[32569]: pgmap v460: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:23:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:07 smithi037 ceph-mon[29195]: pgmap v460: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:23:09.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:09 smithi098 ceph-mon[32569]: pgmap v461: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:23:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:09 smithi037 ceph-mon[29195]: pgmap v461: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:23:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:09 smithi177 ceph-mon[33043]: pgmap v461: 97 pgs: 97 active+clean; 4.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:23:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.513+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1566732906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 msgr2=0x7f2fac10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.513+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1566732906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac10ea00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ca0 tx=0x7f2fa402a950 comp rx=0 tx=0).stop 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1566732906 shutdown_connections 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1566732906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac10ea00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1566732906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103b20 0x7f2fac10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1566732906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103200 0x7f2fac1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1566732906 >> 172.21.15.37:0/1566732906 conn(0x7f2fac0771d0 msgr2=0x7f2fac0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1566732906 shutdown_connections 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1566732906 wait complete. 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 Processor -- start 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 -- start start 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103200 0x7f2fac198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 0x7f2fac198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.514+0000 7f2fb47ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb47ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac112220 con 0x7f2fac10c5d0 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb47ef640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1120a0 con 0x7f2fac103200 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb47ef640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1123a0 con 0x7f2fac103b20 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36424/0 (socket says 172.21.15.37:36424) 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 -- 172.21.15.37:0/1975754039 learned_addr learned my addr 172.21.15.37:0/1975754039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb258b640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103200 0x7f2fac198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 0x7f2fac198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 -- 172.21.15.37:0/1975754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 msgr2=0x7f2fac198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 0x7f2fac198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 -- 172.21.15.37:0/1975754039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103200 msgr2=0x7f2fac198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103200 0x7f2fac198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 -- 172.21.15.37:0/1975754039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fac19d650 con 0x7f2fac10c5d0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 0x7f2fac198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fb2d8c640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.515+0000 7f2fa37fe640 1 -- 172.21.15.37:0/1975754039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4034040 con 0x7f2fac10c5d0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.516+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fac19d8e0 con 0x7f2fac10c5d0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.516+0000 7f2fa37fe640 1 -- 172.21.15.37:0/1975754039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa4030850 con 0x7f2fac10c5d0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.516+0000 7f2fa37fe640 1 -- 172.21.15.37:0/1975754039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40046f0 con 0x7f2fac10c5d0 2024-04-25T21:23:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.516+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fac1a51d0 con 0x7f2fac10c5d0 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.518+0000 7f2fa37fe640 1 -- 172.21.15.37:0/1975754039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fa4037020 con 0x7f2fac10c5d0 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.518+0000 7f2fb75d7740 1 Processor -- start 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.518+0000 7f2fb75d7740 1 -- start start 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca27456d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2748000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556ca269b970 con 0x556ca273aa30 2024-04-25T21:23:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556ca26384a0 con 0x556ca2745c10 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556ca2697930 con 0x556ca273b350 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fa37fe640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 0x7f2f980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36434/0 (socket says 172.21.15.37:36434) 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2540830125 learned_addr learned my addr 172.21.15.37:0/2540830125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb2d8c640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 0x7f2f980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb258b640 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca27456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2748000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fa37fe640 1 -- 172.21.15.37:0/1975754039 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f2fa40f1700 con 0x7f2fac10c5d0 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2540830125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 msgr2=0x556ca27456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca27456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2540830125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 msgr2=0x556ca2748000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2748000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2540830125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca23ae210 con 0x556ca273aa30 2024-04-25T21:23:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0058c0 tx=0x7f2f9c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd65320207da4a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.519+0000 7f2fb2d8c640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 0x7f2f980a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40b9bb0 tx=0x7f2fa403c040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fa0ff9640 1 -- 172.21.15.37:0/2540830125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c019030 con 0x556ca273aa30 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fa0ff9640 1 -- 172.21.15.37:0/2540830125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c0035f0 con 0x556ca273aa30 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fa0ff9640 1 -- 172.21.15.37:0/2540830125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c011720 con 0x556ca273aa30 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2540830125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 msgr2=0x556ca273ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0058c0 tx=0x7f2f9c00b6b0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2540830125 shutdown_connections 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2748000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca27456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2540830125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca273ae10 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2540830125 >> 172.21.15.37:0/2540830125 conn(0x556ca2736f40 msgr2=0x556ca2737320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.520+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2540830125 shutdown_connections 2024-04-25T21:23:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2540830125 wait complete. 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 -- start start 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 Processor -- start 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273aa30 0x556ca2745560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2742db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556ca2748840 con 0x556ca273b350 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556ca26360b0 con 0x556ca2745c10 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556ca27489c0 con 0x556ca273aa30 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb75d7740 10 client.0 _ll_register_callbacks cb 0x556ca2623d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb258b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb258b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36436/0 (socket says 172.21.15.37:36436) 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb258b640 1 -- 172.21.15.37:0/1785772003 learned_addr learned my addr 172.21.15.37:0/1785772003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.521+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273aa30 0x556ca2745560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2742db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 -- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273aa30 msgr2=0x556ca2745560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273aa30 0x556ca2745560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 -- 172.21.15.37:0/1785772003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 msgr2=0x556ca2742db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2742db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 -- 172.21.15.37:0/1785772003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca247a620 con 0x556ca273b350 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2742db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb258b640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bef0 tx=0x7f2fa8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98aea9f383b42b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8005220 con 0x556ca273b350 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f2fa80053c0 con 0x556ca273b350 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f9c0035f0 con 0x556ca273b350 2024-04-25T21:23:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fa80055c0 con 0x556ca273b350 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb75d7740 10 client.17103 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ca2754810 con 0x556ca273b350 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.522+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c0035f0 con 0x556ca273b350 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2fa8010b70 con 0x556ca273b350 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 10 client.17103 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 20 client.17103 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca2755cd0 0x556ca27580c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 4 client.17103 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ca26a9390 con 0x556ca2755cd0 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca2755cd0 0x556ca27580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2f95ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.523+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca2755cd0 0x556ca27580c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61447815273f2116 server_cookie=38101ae2f26aab6c in_seq=0 out_seq=0 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2f95ffb640 1 -- 172.21.15.37:0/1785772003 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ca26a9390 con 0x556ca2755cd0 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2f95ffb640 10 client.17103 handle_command_reply: tid=0 2024-04-25T21:23:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 client.17103 shutdown 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 msgr2=0x556ca2742850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bef0 tx=0x7f2fa8012820 comp rx=0 tx=0).stop 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 shutdown_connections 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca2755cd0 0x556ca27580c0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca2742db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2f95ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273b350 0x556ca2742850 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/1785772003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273aa30 0x556ca2745560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 >> 172.21.15.37:0/1785772003 conn(0x556ca2736f40 msgr2=0x556ca273ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 shutdown_connections 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.524+0000 7f2fb75d7740 1 -- 172.21.15.37:0/1785772003 wait complete. 2024-04-25T21:23:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.525+0000 7f2fb75d7740 20 client.17103 trim_cache size 0 max 16384 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.539+0000 7f2fb75d7740 1 Processor -- start 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.539+0000 7f2fb75d7740 1 -- start start 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca273aa30 0x556ca2751700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca273dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ab920 con 0x556ca2745c10 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ad460 con 0x556ca273aa30 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1abff0 con 0x556ca273b350 2024-04-25T21:23:11.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36452/0 (socket says 172.21.15.37:36452) 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2967878878 learned_addr learned my addr 172.21.15.37:0/2967878878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb258b640 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca273dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2967878878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 msgr2=0x556ca273dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca273dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2967878878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca273aa30 msgr2=0x556ca2751700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca273aa30 0x556ca2751700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/2967878878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ca2743120 con 0x556ca2745c10 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005fd0 tx=0x7f2f9c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f0c7b015cc2881 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2f97fff640 1 -- 172.21.15.37:0/2967878878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c019040 con 0x556ca2745c10 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2f97fff640 1 -- 172.21.15.37:0/2967878878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c01a040 con 0x556ca2745c10 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.540+0000 7f2f97fff640 1 -- 172.21.15.37:0/2967878878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c003f20 con 0x556ca2745c10 2024-04-25T21:23:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2967878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 msgr2=0x556ca2745090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c005fd0 tx=0x7f2f9c00b6b0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2967878878 shutdown_connections 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca2745c10 0x556ca2745090 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca273dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/2967878878 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca273aa30 0x556ca2751700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2967878878 >> 172.21.15.37:0/2967878878 conn(0x556ca2736f40 msgr2=0x556ca273d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2967878878 shutdown_connections 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- 172.21.15.37:0/2967878878 wait complete. 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 -- start start 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.541+0000 7f2fb75d7740 1 Processor -- start 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca2749290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca27497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1acd90 con 0x556ca273aa30 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ac6c0 con 0x556ca2745c10 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1a9940 con 0x556ca273b350 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 10 client.0 _ll_register_callbacks cb 0x7f2fa800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36458/0 (socket says 172.21.15.37:36458) 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/721153081 learned_addr learned my addr 172.21.15.37:0/721153081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 msgr2=0x556ca2749290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca2749290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/721153081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 msgr2=0x556ca27497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca27497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 -- 172.21.15.37:0/721153081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ca2749fd0 con 0x556ca273aa30 2024-04-25T21:23:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb1d8a640 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0058c0 tx=0x7f2f9c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a1a3856d695b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c019040 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f2f9c005020 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f9c01a040 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.542+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2f9c005220 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 10 client.17109 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556ca27bc180 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c01a040 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2f9c02ece0 con 0x556ca273aa30 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 10 client.17109 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 20 client.17109 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca27bc550 0x556ca27be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 4 client.17109 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ca26a3200 con 0x556ca27bc550 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.543+0000 7f2fb258b640 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca27bc550 0x556ca27be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.544+0000 7f2fb258b640 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca27bc550 0x556ca27be940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ced45be277d510 server_cookie=fb10822305615ac8 in_seq=0 out_seq=0 2024-04-25T21:23:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.544+0000 7f2f95ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:11.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2f95ffb640 1 -- 172.21.15.37:0/721153081 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ca26a3200 con 0x556ca27bc550 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2f95ffb640 10 client.17109 handle_command_reply: tid=0 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 client.17109 shutdown 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 msgr2=0x556ca274df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0058c0 tx=0x7f2f9c0037c0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 shutdown_connections 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556ca27bc550 0x556ca27be940 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556ca2745c10 0x556ca27497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556ca273b350 0x556ca2749290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2f95ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 --2- 172.21.15.37:0/721153081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556ca273aa30 0x556ca274df20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 >> 172.21.15.37:0/721153081 conn(0x556ca2736f40 msgr2=0x556ca2731cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 shutdown_connections 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 1 -- 172.21.15.37:0/721153081 wait complete. 2024-04-25T21:23:11.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.558+0000 7f2fb75d7740 20 client.17109 trim_cache size 0 max 16384 2024-04-25T21:23:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:23:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:23:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:23:11.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 msgr2=0x7f2f980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 0x7f2f980a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40b9bb0 tx=0x7f2fa403c040 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 msgr2=0x7f2fac19cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4003f10 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 shutdown_connections 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2fac10c5d0 0x7f2fac19cf20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f980a3dc0 0x7f2f980a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2fac103b20 0x7f2fac198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 --2- 172.21.15.37:0/1975754039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fac103200 0x7f2fac198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 >> 172.21.15.37:0/1975754039 conn(0x7f2fac0771d0 msgr2=0x7f2fac0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 shutdown_connections 2024-04-25T21:23:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:11.559+0000 7f2fb47ef640 1 -- 172.21.15.37:0/1975754039 wait complete. 2024-04-25T21:23:11.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:11 smithi098 ceph-mon[32569]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-04-25T21:23:11.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:11 smithi037 ceph-mon[29195]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-04-25T21:23:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:11 smithi177 ceph-mon[33043]: pgmap v462: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-04-25T21:23:13.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:13 smithi098 ceph-mon[32569]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:23:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:13 smithi037 ceph-mon[29195]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:23:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:13 smithi177 ceph-mon[33043]: pgmap v463: 97 pgs: 97 active+clean; 4.2 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:23:15.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:15 smithi098 ceph-mon[32569]: pgmap v464: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:23:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:15 smithi037 ceph-mon[29195]: pgmap v464: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:23:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:15 smithi177 ceph-mon[33043]: pgmap v464: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:23:17.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:17 smithi098 ceph-mon[32569]: pgmap v465: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:23:17.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:17 smithi037 ceph-mon[29195]: pgmap v465: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:23:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:17 smithi177 ceph-mon[33043]: pgmap v465: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:23:19.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:19 smithi098 ceph-mon[32569]: pgmap v466: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:437/600: open fd = 3 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:438/600: open fd = 3 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.961 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:439/600: open fd = 3 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:440/600: open fd = 3 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:441/600: open fd = 3 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.962 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:442/600: open fd = 3 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:443/600: open fd = 3 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:444/600: open fd = 3 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.963 INFO:tasks.workunit.client.0.smithi037.stdout:445/600: open fd = 3 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:446/600: open fd = 3 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:447/600: open fd = 3 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:448/600: open fd = 3 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.964 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:449/600: open fd = 3 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:450/600: open fd = 3 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:451/600: open fd = 3 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.965 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:452/600: open fd = 3 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:453/600: open fd = 3 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:454/600: open fd = 3 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.966 INFO:tasks.workunit.client.0.smithi037.stdout:455/600: open fd = 3 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:456/600: open fd = 3 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:457/600: open fd = 3 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:458/600: open fd = 3 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.967 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:459/600: open fd = 3 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:460/600: open fd = 3 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:461/600: open fd = 3 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.968 INFO:tasks.workunit.client.0.smithi037.stdout:462/600: open fd = 3 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:463/600: open fd = 3 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:464/600: open fd = 3 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:465/600: open fd = 3 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.969 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:466/600: open fd = 3 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:467/600: open fd = 3 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:468/600: open fd = 3 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.970 INFO:tasks.workunit.client.0.smithi037.stdout:469/600: open fd = 3 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:470/600: open fd = 3 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:471/600: open fd = 3 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:472/600: open fd = 3 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:473/600: open fd = 3 2024-04-25T21:23:19.971 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:474/600: open fd = 3 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:475/600: open fd = 3 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:476/600: open fd = 3 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:477/600: open fd = 3 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.972 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:478/600: open fd = 3 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:479/600: open fd = 3 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:480/600: open fd = 3 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.973 INFO:tasks.workunit.client.0.smithi037.stdout:481/600: open fd = 3 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:482/600: open fd = 3 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:483/600: open fd = 3 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:484/600: open fd = 3 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.974 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:485/600: open fd = 3 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:486/600: open fd = 3 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:487/600: open fd = 3 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:488/600: open fd = 3 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.975 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:489/600: open fd = 3 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:490/600: open fd = 3 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:491/600: open fd = 3 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:492/600: open fd = 3 2024-04-25T21:23:19.976 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:493/600: open fd = 3 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:494/600: open fd = 3 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:495/600: open fd = 3 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.977 INFO:tasks.workunit.client.0.smithi037.stdout:496/600: open fd = 3 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:497/600: open fd = 3 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:498/600: open fd = 3 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:23:19.978 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:23:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:19 smithi037 ceph-mon[29195]: pgmap v466: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T21:23:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:19 smithi177 ceph-mon[33043]: pgmap v466: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-25T21:23:21.890 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:21 smithi177 ceph-mon[33043]: pgmap v467: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-04-25T21:23:21.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:21 smithi098 ceph-mon[32569]: pgmap v467: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-04-25T21:23:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:21 smithi037 ceph-mon[29195]: pgmap v467: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 60 KiB/s wr, 21 op/s 2024-04-25T21:23:23.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:23 smithi098 ceph-mon[32569]: pgmap v468: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:23 smithi037 ceph-mon[29195]: pgmap v468: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:23 smithi177 ceph-mon[33043]: pgmap v468: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.658+0000 7fc001e91640 1 -- 172.21.15.37:0/3246134607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc10c550 msgr2=0x7fbffc10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 --2- 172.21.15.37:0/3246134607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc10c550 0x7fbffc10e9d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009ca0 tx=0x7fbff402a950 comp rx=0 tx=0).stop 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 -- 172.21.15.37:0/3246134607 shutdown_connections 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 --2- 172.21.15.37:0/3246134607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc10c550 0x7fbffc10e9d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 --2- 172.21.15.37:0/3246134607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103aa0 0x7fbffc10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 --2- 172.21.15.37:0/3246134607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc103180 0x7fbffc103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 -- 172.21.15.37:0/3246134607 >> 172.21.15.37:0/3246134607 conn(0x7fbffc077fd0 msgr2=0x7fbffc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 -- 172.21.15.37:0/3246134607 shutdown_connections 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 -- 172.21.15.37:0/3246134607 wait complete. 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 Processor -- start 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.659+0000 7fc001e91640 1 -- start start 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103180 0x7fbffc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 0x7fbffc19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1121c0 con 0x7fbffc103aa0 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc112040 con 0x7fbffc103180 2024-04-25T21:23:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fc001e91640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc112340 con 0x7fbffc10c550 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48752/0 (socket says 172.21.15.37:48752) 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 -- 172.21.15.37:0/593999840 learned_addr learned my addr 172.21.15.37:0/593999840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103180 0x7fbffc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffbfff640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 0x7fbffc19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 -- 172.21.15.37:0/593999840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 msgr2=0x7fbffc19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 0x7fbffc19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 -- 172.21.15.37:0/593999840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103180 msgr2=0x7fbffc198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103180 0x7fbffc198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 -- 172.21.15.37:0/593999840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbffc19d650 con 0x7fbffc103aa0 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffbfff640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 0x7fbffc19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.660+0000 7fbffaffd640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.661+0000 7fbff8ff9640 1 -- 172.21.15.37:0/593999840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0017070 con 0x7fbffc103aa0 2024-04-25T21:23:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.661+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbffc19d940 con 0x7fbffc103aa0 2024-04-25T21:23:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.661+0000 7fbff8ff9640 1 -- 172.21.15.37:0/593999840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff0009c30 con 0x7fbffc103aa0 2024-04-25T21:23:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.661+0000 7fbff8ff9640 1 -- 172.21.15.37:0/593999840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff000f620 con 0x7fbffc103aa0 2024-04-25T21:23:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.661+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbffc1a51d0 con 0x7fbffc103aa0 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.663+0000 7fbff8ff9640 1 -- 172.21.15.37:0/593999840 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbff0010460 con 0x7fbffc103aa0 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.663+0000 7fc004bab740 1 Processor -- start 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.663+0000 7fc004bab740 1 -- start start 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbff8ff9640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 0x7fbfe40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a179a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a186bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b64a0de970 con 0x55b64a179f40 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b64a07b4a0 con 0x55b64a184800 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fc004bab740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b64a0da930 con 0x55b64a179620 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffbfff640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 0x7fbfe40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbff8ff9640 1 -- 172.21.15.37:0/593999840 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fbff00d03b0 con 0x7fbffc103aa0 2024-04-25T21:23:24.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48754/0 (socket says 172.21.15.37:48754) 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 -- 172.21.15.37:0/3774403010 learned_addr learned my addr 172.21.15.37:0/3774403010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a179a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffbfff640 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a186bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 -- 172.21.15.37:0/3774403010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 msgr2=0x55b64a179a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a179a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 -- 172.21.15.37:0/3774403010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 msgr2=0x55b64a186bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a186bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 -- 172.21.15.37:0/3774403010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b649df1210 con 0x55b64a179f40 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.664+0000 7fbffaffd640 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000c00 tx=0x7fbff009bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14a80dd9f80d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fbffbfff640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 0x7fbfe40a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005cc0 tx=0x7fbff402f950 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fbfe1ffb640 1 -- 172.21.15.37:0/3774403010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0020040 con 0x55b64a179f40 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fbfe1ffb640 1 -- 172.21.15.37:0/3774403010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff0009de0 con 0x55b64a179f40 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fbfe1ffb640 1 -- 172.21.15.37:0/3774403010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00989c0 con 0x55b64a179f40 2024-04-25T21:23:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 -- 172.21.15.37:0/3774403010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 msgr2=0x55b64a1842c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000c00 tx=0x7fbff009bfb0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 -- 172.21.15.37:0/3774403010 shutdown_connections 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a186bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a1842c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 --2- 172.21.15.37:0/3774403010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a179a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 -- 172.21.15.37:0/3774403010 >> 172.21.15.37:0/3774403010 conn(0x55b64a175b30 msgr2=0x55b64a175f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 -- 172.21.15.37:0/3774403010 shutdown_connections 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.665+0000 7fc004bab740 1 -- 172.21.15.37:0/3774403010 wait complete. 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 -- start start 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 Processor -- start 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a1841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179f40 0x55b64a1814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b64a1873d0 con 0x55b64a184800 2024-04-25T21:23:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b64a0790b0 con 0x55b64a179f40 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55b64a187550 con 0x55b64a179620 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fc004bab740 10 client.0 _ll_register_callbacks cb 0x55b64a066d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.666+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48768/0 (socket says 172.21.15.37:48768) 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 -- 172.21.15.37:0/3177665585 learned_addr learned my addr 172.21.15.37:0/3177665585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a1841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffaffd640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179f40 0x55b64a1814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 -- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 msgr2=0x55b64a1841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a1841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 -- 172.21.15.37:0/3177665585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179f40 msgr2=0x55b64a1814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179f40 0x55b64a1814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 -- 172.21.15.37:0/3177665585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b649ebd620 con 0x55b64a184800 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbffbfff640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbff4035000 tx=0x7fbff402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b6b7a730cc3648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbfdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4034040 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fbff4004240 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbff0009de0 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbff4058050 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fc004bab740 10 client.17121 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.667+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b64a193360 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff0009de0 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbff403d960 con 0x55b64a184800 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 10 client.17121 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 20 client.17121 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a194870 0x55b64a196c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 4 client.17121 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b64a0ec390 con 0x55b64a194870 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.668+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a194870 0x55b64a196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.669+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a194870 0x55b64a196c60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84023166c7ec0e14 server_cookie=6119b09336c9ef56 in_seq=0 out_seq=0 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.669+0000 7fbfdaffd640 10 client.17121 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fbfdaffd640 1 -- 172.21.15.37:0/3177665585 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b64a0ec390 con 0x55b64a194870 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fbfdaffd640 10 client.17121 handle_command_reply: tid=0 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 client.17121 shutdown 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 msgr2=0x55b64a181ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbff4035000 tx=0x7fbff402a950 comp rx=0 tx=0).stop 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 shutdown_connections 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a194870 0x55b64a196c60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a184800 0x55b64a181ad0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fbfdaffd640 0 client.17121 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179f40 0x55b64a1814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 --2- 172.21.15.37:0/3177665585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179620 0x55b64a1841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 >> 172.21.15.37:0/3177665585 conn(0x55b64a175b30 msgr2=0x55b64a17b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 shutdown_connections 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 1 -- 172.21.15.37:0/3177665585 wait complete. 2024-04-25T21:23:24.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.670+0000 7fc004bab740 20 client.17121 trim_cache size 0 max 16384 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 Processor -- start 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 -- start start 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 0x55b64a19c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1adb30 con 0x55b64a179620 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1af670 con 0x55b64a184800 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fc004bab740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ae200 con 0x55b64a179f40 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.685+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48776/0 (socket says 172.21.15.37:48776) 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 -- 172.21.15.37:0/2026303664 learned_addr learned my addr 172.21.15.37:0/2026303664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffaffd640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 0x55b64a19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffbfff640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 -- 172.21.15.37:0/2026303664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 msgr2=0x55b64a19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 0x55b64a19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 -- 172.21.15.37:0/2026303664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 msgr2=0x55b64a19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 -- 172.21.15.37:0/2026303664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64a1972e0 con 0x55b64a179620 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffaffd640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 0x55b64a19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55b64a1974b0 tx=0x7fbfec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77597096c4799351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbfe17fa640 1 -- 172.21.15.37:0/2026303664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01f250 con 0x55b64a179620 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbfe17fa640 1 -- 172.21.15.37:0/2026303664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec004490 con 0x55b64a179620 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.686+0000 7fbfe17fa640 1 -- 172.21.15.37:0/2026303664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec010d90 con 0x55b64a179620 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- 172.21.15.37:0/2026303664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 msgr2=0x55b64a190290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55b64a1974b0 tx=0x7fbfec005f90 comp rx=0 tx=0).stop 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- 172.21.15.37:0/2026303664 shutdown_connections 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a184800 0x55b64a19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a179f40 0x55b64a19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- 172.21.15.37:0/2026303664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179620 0x55b64a190290 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- 172.21.15.37:0/2026303664 >> 172.21.15.37:0/2026303664 conn(0x55b64a175b30 msgr2=0x55b64a174cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- 172.21.15.37:0/2026303664 shutdown_connections 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- 172.21.15.37:0/2026303664 wait complete. 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- start start 2024-04-25T21:23:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 Processor -- start 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179620 0x55b64a18a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a184800 0x55b64a1f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1aefa0 con 0x55b64a179f40 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1ae8d0 con 0x55b64a179620 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.687+0000 7fc004bab740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fbffc1abb30 con 0x55b64a184800 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fc004bab740 10 client.0 _ll_register_callbacks cb 0x7fbff4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48786/0 (socket says 172.21.15.37:48786) 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 -- 172.21.15.37:0/1267096551 learned_addr learned my addr 172.21.15.37:0/1267096551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179620 0x55b64a18a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffbfff640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a184800 0x55b64a1f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 -- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a184800 msgr2=0x55b64a1f9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a184800 0x55b64a1f9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 -- 172.21.15.37:0/1267096551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179620 msgr2=0x55b64a18a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179620 0x55b64a18a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 -- 172.21.15.37:0/1267096551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b64a186e20 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbffaffd640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ce100 tx=0x7fbff0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14109c184cd9971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbfdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff00b5060 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fbff00cbc30 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec004490 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.688+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbff007c050 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 10 client.17127 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b64a1820d0 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfec004490 con 0x55b64a179f40 2024-04-25T21:23:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbff00d0030 con 0x55b64a179f40 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 10 client.17127 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 20 client.17127 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a1fb2e0 0x55b64a1fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 4 client.17127 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b64a0e6200 con 0x55b64a1fb2e0 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.689+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a1fb2e0 0x55b64a1fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.690+0000 7fbfdaffd640 10 client.17127 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.690+0000 7fbffb7fe640 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a1fb2e0 0x55b64a1fd6d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af7da3b8851b7e08 server_cookie=229f674493ec8e1d in_seq=0 out_seq=0 2024-04-25T21:23:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fbfdaffd640 1 -- 172.21.15.37:0/1267096551 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b64a0e6200 con 0x55b64a1fb2e0 2024-04-25T21:23:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fbfdaffd640 10 client.17127 handle_command_reply: tid=0 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 client.17127 shutdown 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 msgr2=0x55b64a18aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbff00ce100 tx=0x7fbff0005bd0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 shutdown_connections 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55b64a1fb2e0 0x55b64a1fd6d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b64a184800 0x55b64a1f9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b64a179f40 0x55b64a18aa20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fbfdaffd640 0 client.17127 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 --2- 172.21.15.37:0/1267096551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b64a179620 0x55b64a18a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 >> 172.21.15.37:0/1267096551 conn(0x55b64a175b30 msgr2=0x55b64a18f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 shutdown_connections 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 1 -- 172.21.15.37:0/1267096551 wait complete. 2024-04-25T21:23:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.704+0000 7fc004bab740 20 client.17127 trim_cache size 0 max 16384 2024-04-25T21:23:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:23:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:23:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:23:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 msgr2=0x7fbfe40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 0x7fbfe40a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005cc0 tx=0x7fbff402f950 comp rx=0 tx=0).stop 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 msgr2=0x7fbffc198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbff000d950 tx=0x7fbff000de10 comp rx=0 tx=0).stop 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 shutdown_connections 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fbfe40a3dc0 0x7fbfe40a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbffc10c550 0x7fbffc19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbffc103aa0 0x7fbffc198ba0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 --2- 172.21.15.37:0/593999840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbffc103180 0x7fbffc198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 >> 172.21.15.37:0/593999840 conn(0x7fbffc077fd0 msgr2=0x7fbffc0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 shutdown_connections 2024-04-25T21:23:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:24.705+0000 7fc001e91640 1 -- 172.21.15.37:0/593999840 wait complete. 2024-04-25T21:23:25.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:25 smithi098 ceph-mon[32569]: pgmap v469: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:25 smithi037 ceph-mon[29195]: pgmap v469: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:25 smithi177 ceph-mon[33043]: pgmap v469: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:27.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:27 smithi098 ceph-mon[32569]: pgmap v470: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:27 smithi037 ceph-mon[29195]: pgmap v470: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:27 smithi177 ceph-mon[33043]: pgmap v470: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:29.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:29 smithi098 ceph-mon[32569]: pgmap v471: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:29 smithi037 ceph-mon[29195]: pgmap v471: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:29 smithi177 ceph-mon[33043]: pgmap v471: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-25T21:23:31.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:31 smithi098 ceph-mon[32569]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:31 smithi037 ceph-mon[29195]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:31 smithi177 ceph-mon[33043]: pgmap v472: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:23:33.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:33 smithi098 ceph-mon[32569]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:23:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:33 smithi037 ceph-mon[29195]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:23:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:33 smithi177 ceph-mon[33043]: pgmap v473: 97 pgs: 97 active+clean; 5.0 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:23:35.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:35 smithi098 ceph-mon[32569]: pgmap v474: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:35.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:35 smithi037 ceph-mon[29195]: pgmap v474: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:35 smithi177 ceph-mon[33043]: pgmap v474: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 -- 172.21.15.37:0/4240688135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c101aa0 msgr2=0x7f0f0c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 --2- 172.21.15.37:0/4240688135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c101aa0 0x7f0f0c105a90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009900 tx=0x7f0f0002a410 comp rx=0 tx=0).stop 2024-04-25T21:23:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 -- 172.21.15.37:0/4240688135 shutdown_connections 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 --2- 172.21.15.37:0/4240688135 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c106170 0x7f0f0c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 --2- 172.21.15.37:0/4240688135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c101aa0 0x7f0f0c105a90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 --2- 172.21.15.37:0/4240688135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c1010f0 0x7f0f0c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 -- 172.21.15.37:0/4240688135 >> 172.21.15.37:0/4240688135 conn(0x7f0f0c077fd0 msgr2=0x7f0f0c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 -- 172.21.15.37:0/4240688135 shutdown_connections 2024-04-25T21:23:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.805+0000 7f0f10e23640 1 -- 172.21.15.37:0/4240688135 wait complete. 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 Processor -- start 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 -- start start 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c1010f0 0x7f0f0c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c199fa0 0x7f0f0c1a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1110e0 con 0x7f0f0c106170 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c110f60 con 0x7f0f0c1010f0 2024-04-25T21:23:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.806+0000 7f0f10e23640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c111260 con 0x7f0f0c199fa0 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55528/0 (socket says 172.21.15.37:55528) 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 -- 172.21.15.37:0/93815134 learned_addr learned my addr 172.21.15.37:0/93815134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c199fa0 0x7f0f0c1a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c1010f0 0x7f0f0c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 -- 172.21.15.37:0/93815134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c199fa0 msgr2=0x7f0f0c1a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c199fa0 0x7f0f0c1a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 -- 172.21.15.37:0/93815134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c1010f0 msgr2=0x7f0f0c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c1010f0 0x7f0f0c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 -- 172.21.15.37:0/93815134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c1a4d40 con 0x7f0f0c106170 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005ab0 tx=0x7f0f00003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0efb7fe640 1 -- 172.21.15.37:0/93815134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0003f070 con 0x7f0f0c106170 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c1a4fd0 con 0x7f0f0c106170 2024-04-25T21:23:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0efb7fe640 1 -- 172.21.15.37:0/93815134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f00031d60 con 0x7f0f0c106170 2024-04-25T21:23:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0efb7fe640 1 -- 172.21.15.37:0/93815134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00035030 con 0x7f0f0c106170 2024-04-25T21:23:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.807+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1a5430 con 0x7f0f0c106170 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0efb7fe640 1 -- 172.21.15.37:0/93815134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f00042070 con 0x7f0f0c106170 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 Processor -- start 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 -- start start 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4b620 0x55a5a0f4ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f562c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0eb0970 con 0x55a5a0f56800 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0e4d4a0 con 0x55a5a0f4b620 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.810+0000 7f0f13c11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0eac930 con 0x55a5a0f4bf40 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0efb7fe640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 0x7f0ef40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55532/0 (socket says 172.21.15.37:55532) 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 -- 172.21.15.37:0/3390094910 learned_addr learned my addr 172.21.15.37:0/3390094910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4b620 0x55a5a0f4ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 0x7f0ef40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0efb7fe640 1 -- 172.21.15.37:0/93815134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f0f000f0ed0 con 0x7f0f0c106170 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 -- 172.21.15.37:0/3390094910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 msgr2=0x55a5a0f562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 -- 172.21.15.37:0/3390094910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4b620 msgr2=0x55a5a0f4ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4b620 0x55a5a0f4ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 -- 172.21.15.37:0/3390094910 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a0bc3210 con 0x55a5a0f56800 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0f000b98f0 tx=0x7f0f0003b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb2b38547890b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 0x7f0ef40a62b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f04012890 tx=0x7f0f0400b2e0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0ef8ff9640 1 -- 172.21.15.37:0/3390094910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00049070 con 0x55a5a0f56800 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0ef8ff9640 1 -- 172.21.15.37:0/3390094910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f000c1220 con 0x55a5a0f56800 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.811+0000 7f0ef8ff9640 1 -- 172.21.15.37:0/3390094910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0003b4f0 con 0x55a5a0f56800 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 -- 172.21.15.37:0/3390094910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 msgr2=0x55a5a0f58bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f0f000b98f0 tx=0x7f0f0003b890 comp rx=0 tx=0).stop 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 -- 172.21.15.37:0/3390094910 shutdown_connections 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f58bf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 --2- 172.21.15.37:0/3390094910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4b620 0x55a5a0f4ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 -- 172.21.15.37:0/3390094910 >> 172.21.15.37:0/3390094910 conn(0x55a5a0f47b30 msgr2=0x55a5a0f47f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 -- 172.21.15.37:0/3390094910 shutdown_connections 2024-04-25T21:23:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.812+0000 7f0f13c11740 1 -- 172.21.15.37:0/3390094910 wait complete. 2024-04-25T21:23:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 -- start start 2024-04-25T21:23:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 Processor -- start 2024-04-25T21:23:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4b620 0x55a5a0f56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f53420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0eb0970 con 0x55a5a0f56800 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0e4b0b0 con 0x55a5a0f4bf40 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55a5a0f591f0 con 0x55a5a0f4b620 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f13c11740 10 client.0 _ll_register_callbacks cb 0x55a5a0e38d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55538/0 (socket says 172.21.15.37:55538) 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 -- 172.21.15.37:0/1545910957 learned_addr learned my addr 172.21.15.37:0/1545910957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4b620 0x55a5a0f56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f53420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 -- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4b620 msgr2=0x55a5a0f56130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4b620 0x55a5a0f56130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.813+0000 7f0f0ad9d640 1 -- 172.21.15.37:0/1545910957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 msgr2=0x55a5a0f53420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f53420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f0ad9d640 1 -- 172.21.15.37:0/1545910957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a0c8f620 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f53420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0f040043c0 tx=0x7f0f040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628dd7ca926ead5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04025030 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f0f0401b070 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f000c1220 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f04005020 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 10 client.17139 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5a0f64fc0 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f000c1220 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f0401e050 con 0x55a5a0f56800 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 10 client.17139 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.814+0000 7f0f13c11740 20 client.17139 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0f664a0 0x55a5a0f68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0f13c11740 4 client.17139 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5a0ebe390 con 0x55a5a0f664a0 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0f664a0 0x55a5a0f68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0ef1ffb640 10 client.17139 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.815+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0f664a0 0x55a5a0f68890 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57d2c7f03f6a110 server_cookie=af58f7ff36a1edc3 in_seq=0 out_seq=0 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1545910957 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5a0ebe390 con 0x55a5a0f664a0 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0ef1ffb640 10 client.17139 handle_command_reply: tid=0 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 client.17139 shutdown 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 msgr2=0x55a5a0f53960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f0f040043c0 tx=0x7f0f040046c0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 shutdown_connections 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0f664a0 0x55a5a0f68890 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f56800 0x55a5a0f53960 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f53420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1545910957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4b620 0x55a5a0f56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0ef1ffb640 0 client.17139 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 >> 172.21.15.37:0/1545910957 conn(0x55a5a0f47b30 msgr2=0x55a5a0f4d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 shutdown_connections 2024-04-25T21:23:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.816+0000 7f0f13c11740 1 -- 172.21.15.37:0/1545910957 wait complete. 2024-04-25T21:23:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.817+0000 7f0f13c11740 20 client.17139 trim_cache size 0 max 16384 2024-04-25T21:23:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.831+0000 7f0f13c11740 1 Processor -- start 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.831+0000 7f0f13c11740 1 -- start start 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f46b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f6e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1adba0 con 0x55a5a0f4b620 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1af6e0 con 0x55a5a0f56800 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f13c11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ae270 con 0x55a5a0f4bf40 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55552/0 (socket says 172.21.15.37:55552) 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 -- 172.21.15.37:0/436771990 learned_addr learned my addr 172.21.15.37:0/436771990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f46b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 -- 172.21.15.37:0/436771990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 msgr2=0x55a5a0f46b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f46b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 -- 172.21.15.37:0/436771990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 msgr2=0x55a5a0f6e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f6e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 -- 172.21.15.37:0/436771990 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a0f4c9f0 con 0x55a5a0f4b620 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009950 tx=0x7f0efc014bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7269eff5486bbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0ef3fff640 1 -- 172.21.15.37:0/436771990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc0140d0 con 0x55a5a0f4b620 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0ef3fff640 1 -- 172.21.15.37:0/436771990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc014270 con 0x55a5a0f4b620 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.832+0000 7f0ef3fff640 1 -- 172.21.15.37:0/436771990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc01e290 con 0x55a5a0f4b620 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- 172.21.15.37:0/436771990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 msgr2=0x55a5a0f61f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009950 tx=0x7f0efc014bd0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- 172.21.15.37:0/436771990 shutdown_connections 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f6e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f46b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 --2- 172.21.15.37:0/436771990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f61f30 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- 172.21.15.37:0/436771990 >> 172.21.15.37:0/436771990 conn(0x55a5a0f47b30 msgr2=0x55a5a0f46420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- 172.21.15.37:0/436771990 shutdown_connections 2024-04-25T21:23:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- 172.21.15.37:0/436771990 wait complete. 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 -- start start 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.833+0000 7f0f13c11740 1 Processor -- start 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f5c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f5c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1af010 con 0x55a5a0f4b620 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1ae940 con 0x55a5a0f56800 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1abba0 con 0x55a5a0f4bf40 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f13c11740 10 client.0 _ll_register_callbacks cb 0x7f0f0400bc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55558/0 (socket says 172.21.15.37:55558) 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 -- 172.21.15.37:0/1971513017 learned_addr learned my addr 172.21.15.37:0/1971513017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f5c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f5c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 -- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 msgr2=0x55a5a0f5c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f5c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 -- 172.21.15.37:0/1971513017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 msgr2=0x55a5a0f5c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f5c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 -- 172.21.15.37:0/1971513017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a0fcbb40 con 0x55a5a0f4b620 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0ad9d640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f5c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0f0a59c640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0141f0 tx=0x7f0efc009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671f9f68fdb88160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.834+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc011b10 con 0x55a5a0f4b620 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7f0efc004a40 con 0x55a5a0f4b620 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc014270 con 0x55a5a0f4b620 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0efc011cb0 con 0x55a5a0f4b620 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 10 client.17145 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a5a0fcce50 con 0x55a5a0f4b620 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc014270 con 0x55a5a0f4b620 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0efc00f040 con 0x55a5a0f4b620 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 10 client.17145 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 20 client.17145 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0fcd540 0x55a5a0fcf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 4 client.17145 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5a0f53cf0 con 0x55a5a0fcd540 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.835+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0fcd540 0x55a5a0fcf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.836+0000 7f0f09d9b640 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0fcd540 0x55a5a0fcf960 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3dc0313732e9368 server_cookie=57af9686b2c9f891 in_seq=0 out_seq=0 2024-04-25T21:23:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.836+0000 7f0ef1ffb640 10 client.17145 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0ef1ffb640 1 -- 172.21.15.37:0/1971513017 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5a0f53cf0 con 0x55a5a0fcd540 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0ef1ffb640 10 client.17145 handle_command_reply: tid=0 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 client.17145 shutdown 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 msgr2=0x55a5a0f5bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0141f0 tx=0x7f0efc009cf0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 shutdown_connections 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55a5a0fcd540 0x55a5a0fcf960 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a5a0f56800 0x55a5a0f5c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a5a0f4bf40 0x55a5a0f5c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0ef1ffb640 0 client.17145 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 --2- 172.21.15.37:0/1971513017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a5a0f4b620 0x55a5a0f5bf60 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 >> 172.21.15.37:0/1971513017 conn(0x55a5a0f47b30 msgr2=0x55a5a0f46cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 shutdown_connections 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 1 -- 172.21.15.37:0/1971513017 wait complete. 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.850+0000 7f0f13c11740 20 client.17145 trim_cache size 0 max 16384 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:23:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:23:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:23:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 msgr2=0x7f0ef40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 0x7f0ef40a62b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f04012890 tx=0x7f0f0400b2e0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 msgr2=0x7f0f0c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f0f00005ab0 tx=0x7f0f00003cd0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 shutdown_connections 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f0ef40a3df0 0x7f0ef40a62b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f0c199fa0 0x7f0f0c1a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f0c106170 0x7f0f0c199a60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 --2- 172.21.15.37:0/93815134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f0c1010f0 0x7f0f0c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 >> 172.21.15.37:0/93815134 conn(0x7f0f0c077fd0 msgr2=0x7f0f0c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 shutdown_connections 2024-04-25T21:23:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:37.851+0000 7f0f10e23640 1 -- 172.21.15.37:0/93815134 wait complete. 2024-04-25T21:23:37.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:37 smithi098 ceph-mon[32569]: pgmap v475: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:37.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:37 smithi037 ceph-mon[29195]: pgmap v475: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:37 smithi177 ceph-mon[33043]: pgmap v475: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:39.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:39 smithi098 ceph-mon[32569]: pgmap v476: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:39.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:39 smithi037 ceph-mon[29195]: pgmap v476: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:39 smithi177 ceph-mon[33043]: pgmap v476: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:41.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:41 smithi098 ceph-mon[32569]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-25T21:23:41.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:41 smithi037 ceph-mon[29195]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-25T21:23:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:41 smithi177 ceph-mon[33043]: pgmap v477: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-25T21:23:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:43 smithi098 ceph-mon[32569]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T21:23:43.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:43 smithi037 ceph-mon[29195]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T21:23:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:43 smithi177 ceph-mon[33043]: pgmap v478: 97 pgs: 97 active+clean; 1.5 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-25T21:23:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:45 smithi098 ceph-mon[32569]: pgmap v479: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:45 smithi037 ceph-mon[29195]: pgmap v479: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:45 smithi177 ceph-mon[33043]: pgmap v479: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-25T21:23:47.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:47 smithi098 ceph-mon[32569]: pgmap v480: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:47 smithi037 ceph-mon[29195]: pgmap v480: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:47 smithi177 ceph-mon[33043]: pgmap v480: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:49 smithi098 ceph-mon[32569]: pgmap v481: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:23:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:23:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:23:49.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:23:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:49 smithi037 ceph-mon[29195]: pgmap v481: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:23:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:23:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:23:49.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:23:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:49 smithi177 ceph-mon[33043]: pgmap v481: 97 pgs: 97 active+clean; 1.7 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:23:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:23:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:23:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:23:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 -- 172.21.15.37:0/2008875148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2801016a0 msgr2=0x7fd28010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 --2- 172.21.15.37:0/2008875148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2801016a0 0x7fd28010e3f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fd270009900 tx=0x7fd27002a410 comp rx=0 tx=0).stop 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 -- 172.21.15.37:0/2008875148 shutdown_connections 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 --2- 172.21.15.37:0/2008875148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd28010e930 0x7fd280110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 --2- 172.21.15.37:0/2008875148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2801016a0 0x7fd28010e3f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 --2- 172.21.15.37:0/2008875148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd280101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 -- 172.21.15.37:0/2008875148 >> 172.21.15.37:0/2008875148 conn(0x7fd2800771d0 msgr2=0x7fd2800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 -- 172.21.15.37:0/2008875148 shutdown_connections 2024-04-25T21:23:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.952+0000 7fd288013640 1 -- 172.21.15.37:0/2008875148 wait complete. 2024-04-25T21:23:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 Processor -- start 2024-04-25T21:23:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 -- start start 2024-04-25T21:23:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd2801a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2801016a0 0x7fd2801a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd280114440 con 0x7fd28010e930 2024-04-25T21:23:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801142c0 con 0x7fd280100d80 2024-04-25T21:23:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd288013640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801145c0 con 0x7fd2801016a0 2024-04-25T21:23:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59942/0 (socket says 172.21.15.37:59942) 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 -- 172.21.15.37:0/1880466710 learned_addr learned my addr 172.21.15.37:0/1880466710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd285daf640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd2801a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2855ae640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2801016a0 0x7fd2801a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 -- 172.21.15.37:0/1880466710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2801016a0 msgr2=0x7fd2801a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2801016a0 0x7fd2801a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 -- 172.21.15.37:0/1880466710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 msgr2=0x7fd2801a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd2801a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd2865b0640 1 -- 172.21.15.37:0/1880466710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28019afd0 con 0x7fd28010e930 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.953+0000 7fd285daf640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd2801a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd2865b0640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd278009fd0 tx=0x7fd27800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd276ffd640 1 -- 172.21.15.37:0/1880466710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd278004d20 con 0x7fd28010e930 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd276ffd640 1 -- 172.21.15.37:0/1880466710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd278005020 con 0x7fd28010e930 2024-04-25T21:23:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28019b2c0 con 0x7fd28010e930 2024-04-25T21:23:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd276ffd640 1 -- 172.21.15.37:0/1880466710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2780116b0 con 0x7fd28010e930 2024-04-25T21:23:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.954+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2801a74e0 con 0x7fd28010e930 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.956+0000 7fd276ffd640 1 -- 172.21.15.37:0/1880466710 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2780051d0 con 0x7fd28010e930 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.956+0000 7fd28ae06740 1 Processor -- start 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 -- start start 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e55bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5615f4dad970 con 0x5615f4e48620 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615f4d4a4a0 con 0x5615f4e53800 2024-04-25T21:23:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd28ae06740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5615f4da9930 con 0x5615f4e48f40 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd276ffd640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 0x7fd26c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59954/0 (socket says 172.21.15.37:59954) 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 -- 172.21.15.37:0/3770151707 learned_addr learned my addr 172.21.15.37:0/3770151707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd285daf640 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2865b0640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 0x7fd26c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd276ffd640 1 -- 172.21.15.37:0/1880466710 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fd2780d0ca0 con 0x7fd28010e930 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e55bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 -- 172.21.15.37:0/3770151707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 msgr2=0x5615f4e532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 -- 172.21.15.37:0/3770151707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 msgr2=0x5615f4e55bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e55bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.957+0000 7fd2855ae640 1 -- 172.21.15.37:0/3770151707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f4ac0210 con 0x5615f4e48620 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.958+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a930 tx=0x7fd270005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544816f6409823ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.958+0000 7fd2865b0640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 0x7fd26c0a6280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2780990f0 tx=0x7fd27806e6d0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.958+0000 7fd26bfff640 1 -- 172.21.15.37:0/3770151707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270044020 con 0x5615f4e48620 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.958+0000 7fd26bfff640 1 -- 172.21.15.37:0/3770151707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2700317e0 con 0x5615f4e48620 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.958+0000 7fd26bfff640 1 -- 172.21.15.37:0/3770151707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270035040 con 0x5615f4e48620 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- 172.21.15.37:0/3770151707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 msgr2=0x5615f4e48a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a930 tx=0x7fd270005c00 comp rx=0 tx=0).stop 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- 172.21.15.37:0/3770151707 shutdown_connections 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e55bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3770151707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e48a00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- 172.21.15.37:0/3770151707 >> 172.21.15.37:0/3770151707 conn(0x5615f4e44b30 msgr2=0x5615f4e44f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- 172.21.15.37:0/3770151707 shutdown_connections 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- 172.21.15.37:0/3770151707 wait complete. 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 -- start start 2024-04-25T21:23:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.959+0000 7fd28ae06740 1 Processor -- start 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e50470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e509f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5615f4e563d0 con 0x5615f4e48620 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5615f4d480b0 con 0x5615f4e53800 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5615f4e56550 con 0x5615f4e48f40 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd28ae06740 10 client.0 _ll_register_callbacks cb 0x5615f4d35d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59970/0 (socket says 172.21.15.37:59970) 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 -- 172.21.15.37:0/3727680047 learned_addr learned my addr 172.21.15.37:0/3727680047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd285daf640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e50470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 -- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 msgr2=0x5615f4e50470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e50470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 -- 172.21.15.37:0/3727680047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 msgr2=0x5615f4e509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 -- 172.21.15.37:0/3727680047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f4b8c620 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd285daf640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e50470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2855ae640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd270032fd0 tx=0x7fd27002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d7460677124bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.960+0000 7fd2697fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270044070 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fd270004250 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2700317e0 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd270035040 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 10 client.17157 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f4e62320 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2700317e0 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd27004a430 con 0x5615f4e48620 2024-04-25T21:23:50.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 10 client.17157 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 20 client.17157 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4e63670 0x5615f4e65a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 4 client.17157 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.961+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f4dbb390 con 0x5615f4e63670 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.962+0000 7fd285daf640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4e63670 0x5615f4e65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.962+0000 7fd2697fa640 10 client.17157 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.962+0000 7fd285daf640 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4e63670 0x5615f4e65a60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4e6a043aec5608 server_cookie=b2b2a79d136c0ba in_seq=0 out_seq=0 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd2697fa640 1 -- 172.21.15.37:0/3727680047 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615f4dbb390 con 0x5615f4e63670 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd2697fa640 10 client.17157 handle_command_reply: tid=0 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 client.17157 shutdown 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 msgr2=0x5615f4e53180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fd270032fd0 tx=0x7fd27002a410 comp rx=0 tx=0).stop 2024-04-25T21:23:50.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 shutdown_connections 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4e63670 0x5615f4e65a60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e53800 0x5615f4e509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e48f40 0x5615f4e50470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 --2- 172.21.15.37:0/3727680047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e53180 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd2697fa640 0 client.17157 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 >> 172.21.15.37:0/3727680047 conn(0x5615f4e44b30 msgr2=0x5615f4e4a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 shutdown_connections 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 1 -- 172.21.15.37:0/3727680047 wait complete. 2024-04-25T21:23:50.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.963+0000 7fd28ae06740 20 client.17157 trim_cache size 0 max 16384 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 Processor -- start 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 -- start start 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801afd10 con 0x5615f4e48620 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801b1850 con 0x5615f4e48f40 2024-04-25T21:23:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.978+0000 7fd28ae06740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801b03e0 con 0x5615f4e53800 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59982/0 (socket says 172.21.15.37:59982) 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 -- 172.21.15.37:0/4098663733 learned_addr learned my addr 172.21.15.37:0/4098663733 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 -- 172.21.15.37:0/4098663733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 msgr2=0x5615f4e68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 -- 172.21.15.37:0/4098663733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 msgr2=0x5615f4e52cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 -- 172.21.15.37:0/4098663733 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f4e5fdc0 con 0x5615f4e48620 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd2855ae640 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd270001eb0 tx=0x7fd270031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90acb2ce31a8330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd26b7fe640 1 -- 172.21.15.37:0/4098663733 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270044040 con 0x5615f4e48620 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd26b7fe640 1 -- 172.21.15.37:0/4098663733 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27004ae00 con 0x5615f4e48620 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.979+0000 7fd26b7fe640 1 -- 172.21.15.37:0/4098663733 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27004a430 con 0x5615f4e48620 2024-04-25T21:23:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- 172.21.15.37:0/4098663733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 msgr2=0x5615f4e5eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fd270001eb0 tx=0x7fd270031400 comp rx=0 tx=0).stop 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- 172.21.15.37:0/4098663733 shutdown_connections 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e68a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e52cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 --2- 172.21.15.37:0/4098663733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e5eeb0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- 172.21.15.37:0/4098663733 >> 172.21.15.37:0/4098663733 conn(0x5615f4e44b30 msgr2=0x5615f4e5e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- 172.21.15.37:0/4098663733 shutdown_connections 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- 172.21.15.37:0/4098663733 wait complete. 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 -- start start 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.980+0000 7fd28ae06740 1 Processor -- start 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e66a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801b1180 con 0x5615f4e48620 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801b0ab0 con 0x5615f4e48f40 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd2801add10 con 0x5615f4e53800 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 10 client.0 _ll_register_callbacks cb 0x7fd270032c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59998/0 (socket says 172.21.15.37:59998) 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 -- 172.21.15.37:0/2643346430 learned_addr learned my addr 172.21.15.37:0/2643346430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd285daf640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e66a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 -- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 msgr2=0x5615f4e66f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 -- 172.21.15.37:0/2643346430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 msgr2=0x5615f4e66a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e66a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 -- 172.21.15.37:0/2643346430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f4ec8850 con 0x5615f4e48620 2024-04-25T21:23:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2855ae640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a910 tx=0x7fd270005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733b229f91d15f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2697fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd270062070 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8513+0+0 (secure 0 0 0) 0x7fd27004a430 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.981+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd27004ae00 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd270044040 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 10 client.17163 fetch_fsmap learned FSMap version 27 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f4ec9b40 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd27004ae00 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd2700037b0 con 0x5615f4e48620 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 10 client.17163 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 20 client.17163 populate_metadata read hostname 'smithi037' 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4eca010 0x5615f4ecc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:23:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 4 client.17163 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:23:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615f4db5200 con 0x5615f4eca010 2024-04-25T21:23:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.982+0000 7fd285daf640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4eca010 0x5615f4ecc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:23:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.983+0000 7fd2697fa640 10 client.17163 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.983+0000 7fd285daf640 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4eca010 0x5615f4ecc400 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99795b4876cce56d server_cookie=ddc7600a21ca9684 in_seq=0 out_seq=0 2024-04-25T21:23:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd2697fa640 1 -- 172.21.15.37:0/2643346430 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615f4db5200 con 0x5615f4eca010 2024-04-25T21:23:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd2697fa640 10 client.17163 handle_command_reply: tid=0 2024-04-25T21:23:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 client.17163 shutdown 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 msgr2=0x5615f4e664c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd27002a910 tx=0x7fd270005850 comp rx=0 tx=0).stop 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 shutdown_connections 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5615f4eca010 0x5615f4ecc400 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615f4e53800 0x5615f4e66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5615f4e48f40 0x5615f4e66a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 --2- 172.21.15.37:0/2643346430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5615f4e48620 0x5615f4e664c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 >> 172.21.15.37:0/2643346430 conn(0x5615f4e44b30 msgr2=0x5615f4e43cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd2697fa640 0 client.17163 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 shutdown_connections 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.996+0000 7fd28ae06740 1 -- 172.21.15.37:0/2643346430 wait complete. 2024-04-25T21:23:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd28ae06740 20 client.17163 trim_cache size 0 max 16384 2024-04-25T21:23:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:23:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:23:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:23:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 msgr2=0x7fd26c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 0x7fd26c0a6280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd2780990f0 tx=0x7fd27806e6d0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 msgr2=0x7fd28019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd278009fd0 tx=0x7fd27800ee60 comp rx=0 tx=0).stop 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 shutdown_connections 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd28010e930 0x7fd28019a800 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd26c0a3dc0 0x7fd26c0a6280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2801016a0 0x7fd2801a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 --2- 172.21.15.37:0/1880466710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd280100d80 0x7fd2801a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:23:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 >> 172.21.15.37:0/1880466710 conn(0x7fd2800771d0 msgr2=0x7fd28010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:23:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 shutdown_connections 2024-04-25T21:23:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:23:50.997+0000 7fd288013640 1 -- 172.21.15.37:0/1880466710 wait complete. 2024-04-25T21:23:51.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:51 smithi098 ceph-mon[32569]: pgmap v482: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:23:51.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:51 smithi037 ceph-mon[29195]: pgmap v482: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:23:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:51 smithi177 ceph-mon[33043]: pgmap v482: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-25T21:23:53.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:53 smithi098 ceph-mon[32569]: pgmap v483: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:23:53.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:53 smithi037 ceph-mon[29195]: pgmap v483: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:23:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:53 smithi177 ceph-mon[33043]: pgmap v483: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-25T21:23:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:55 smithi037 ceph-mon[29195]: pgmap v484: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:23:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:55 smithi177 ceph-mon[33043]: pgmap v484: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:23:56.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:55 smithi098 ceph-mon[32569]: pgmap v484: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-25T21:23:57.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:57 smithi037 ceph-mon[29195]: pgmap v485: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:23:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:57 smithi177 ceph-mon[33043]: pgmap v485: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:23:58.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:57 smithi098 ceph-mon[32569]: pgmap v485: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:23:59.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:23:59 smithi037 ceph-mon[29195]: pgmap v486: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:24:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:23:59 smithi177 ceph-mon[33043]: pgmap v486: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:24:00.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:23:59 smithi098 ceph-mon[32569]: pgmap v486: 97 pgs: 97 active+clean; 2.0 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-25T21:24:01.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:01 smithi037 ceph-mon[29195]: pgmap v487: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:01 smithi177 ceph-mon[33043]: pgmap v487: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:01 smithi098 ceph-mon[32569]: pgmap v487: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:02.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:02 smithi037 ceph-mon[29195]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T21:24:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:02 smithi177 ceph-mon[33043]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T21:24:03.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:02 smithi098 ceph-mon[32569]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T21:24:03.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:03 smithi037 ceph-mon[29195]: pgmap v489: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:24:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:03 smithi177 ceph-mon[33043]: pgmap v489: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:24:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/95140547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e981061b0 msgr2=0x7f2e9810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/95140547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e981061b0 0x7f2e9810dd80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e90009ca0 tx=0x7f2e9002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/95140547 shutdown_connections 2024-04-25T21:24:04.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/95140547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e981061b0 0x7f2e9810dd80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/95140547 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e98101a90 0x7f2e98105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/95140547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 0x7f2e981014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/95140547 >> 172.21.15.37:0/95140547 conn(0x7f2e980771d0 msgr2=0x7f2e980775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/95140547 shutdown_connections 2024-04-25T21:24:04.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.096+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/95140547 wait complete. 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 Processor -- start 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 -- start start 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 0x7f2e981a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 0x7f2e981a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98111120 con 0x7f2e9819a670 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e98110fa0 con 0x7f2e981061b0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9f4e1640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981112a0 con 0x7f2e981010e0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9da7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9da7e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33364/0 (socket says 172.21.15.37:33364) 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.097+0000 7f2e9da7e640 1 -- 172.21.15.37:0/699087908 learned_addr learned my addr 172.21.15.37:0/699087908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 0x7f2e981a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 0x7f2e981a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 -- 172.21.15.37:0/699087908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 msgr2=0x7f2e981a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 0x7f2e981a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 -- 172.21.15.37:0/699087908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 msgr2=0x7f2e981a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 0x7f2e981a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 -- 172.21.15.37:0/699087908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9819b270 con 0x7f2e9819a670 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 0x7f2e981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9da7e640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e90005cc0 tx=0x7f2e90031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e8e7fc640 1 -- 172.21.15.37:0/699087908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90003b70 con 0x7f2e9819a670 2024-04-25T21:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9819b500 con 0x7f2e9819a670 2024-04-25T21:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e8e7fc640 1 -- 172.21.15.37:0/699087908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e900318f0 con 0x7f2e9819a670 2024-04-25T21:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e8e7fc640 1 -- 172.21.15.37:0/699087908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9002f950 con 0x7f2e9819a670 2024-04-25T21:24:04.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.098+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e981a76f0 con 0x7f2e9819a670 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.101+0000 7f2e8e7fc640 1 -- 172.21.15.37:0/699087908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e9002fb50 con 0x7f2e9819a670 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 Processor -- start 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 -- start start 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bcea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561889b33970 con 0x561889bd9800 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561889ad04a0 con 0x561889bce620 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561889b2f930 con 0x561889bcef40 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e8e7fc640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 0x7f2e840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33378/0 (socket says 172.21.15.37:33378) 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2132093514 learned_addr learned my addr 172.21.15.37:0/2132093514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bcea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9da7e640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 0x7f2e840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2132093514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 msgr2=0x561889bd92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2132093514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 msgr2=0x561889bcea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.102+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bcea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2132093514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561889846210 con 0x561889bd9800 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e8e7fc640 1 -- 172.21.15.37:0/699087908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8557+0+0 (secure 0 0 0) 0x7f2e900f1b10 con 0x7f2e9819a670 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e88014880 tx=0x7f2e880095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad29423a2aeb294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e837fe640 1 -- 172.21.15.37:0/2132093514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88014110 con 0x561889bd9800 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e9da7e640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 0x7f2e840a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e900bd840 tx=0x7f2e9002a8c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e837fe640 1 -- 172.21.15.37:0/2132093514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e880142b0 con 0x561889bd9800 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.103+0000 7f2e837fe640 1 -- 172.21.15.37:0/2132093514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800fea0 con 0x561889bd9800 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2132093514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 msgr2=0x561889bdbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e88014880 tx=0x7f2e880095b0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2132093514 shutdown_connections 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bdbbf0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2132093514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bcea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2132093514 >> 172.21.15.37:0/2132093514 conn(0x561889bcab30 msgr2=0x561889bcaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2132093514 shutdown_connections 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2132093514 wait complete. 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 -- start start 2024-04-25T21:24:04.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.104+0000 7f2ea21f3740 1 Processor -- start 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bd9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561889bdc430 con 0x561889bd9800 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561889ace0b0 con 0x561889bce620 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x561889bdc5b0 con 0x561889bcef40 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2ea21f3740 10 client.0 _ll_register_callbacks cb 0x561889abbd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33384/0 (socket says 172.21.15.37:33384) 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2159157102 learned_addr learned my addr 172.21.15.37:0/2159157102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bd9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 msgr2=0x561889bd6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2159157102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 msgr2=0x561889bd9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bd9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/2159157102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561889912620 con 0x561889bd9800 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e88001eb0 tx=0x7f2e88016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f39fecc3428bb7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.105+0000 7f2e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800f760 con 0x561889bd9800 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8557+0+0 (secure 0 0 0) 0x7f2e88014460 con 0x561889bd9800 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e880142b0 con 0x561889bd9800 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e88004020 con 0x561889bd9800 2024-04-25T21:24:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 10 client.17175 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561889be8360 con 0x561889bd9800 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e880142b0 con 0x561889bd9800 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e880046c0 con 0x561889bd9800 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 10 client.17175 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 20 client.17175 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889be97c0 0x561889bebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 4 client.17175 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561889b41390 con 0x561889be97c0 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.106+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889be97c0 0x561889bebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.107+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889be97c0 0x561889bebbb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aab4aa33cb64f4e server_cookie=55331db4c8fb9350 in_seq=0 out_seq=0 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.107+0000 7f2e80ff9640 10 client.17175 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2e80ff9640 1 -- 172.21.15.37:0/2159157102 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561889b41390 con 0x561889be97c0 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2e80ff9640 10 client.17175 handle_command_reply: tid=0 2024-04-25T21:24:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 client.17175 shutdown 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 msgr2=0x561889bd69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e88001eb0 tx=0x7f2e88016040 comp rx=0 tx=0).stop 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 shutdown_connections 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889be97c0 0x561889bebbb0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bd9800 0x561889bd69c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bcef40 0x561889bd6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2e80ff9640 0 client.17175 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/2159157102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889bd9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 >> 172.21.15.37:0/2159157102 conn(0x561889bcab30 msgr2=0x561889bd0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 shutdown_connections 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 1 -- 172.21.15.37:0/2159157102 wait complete. 2024-04-25T21:24:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.108+0000 7f2ea21f3740 20 client.17175 trim_cache size 0 max 16384 2024-04-25T21:24:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.123+0000 7f2ea21f3740 1 Processor -- start 2024-04-25T21:24:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.123+0000 7f2ea21f3740 1 -- start start 2024-04-25T21:24:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.123+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 0x561889bf12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bf1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981afd40 con 0x561889bce620 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b1880 con 0x561889bcef40 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b0410 con 0x561889bd9800 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33392/0 (socket says 172.21.15.37:33392) 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/4107414558 learned_addr learned my addr 172.21.15.37:0/4107414558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 0x561889bf12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bf1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/4107414558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 msgr2=0x561889bf1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bf1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/4107414558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 msgr2=0x561889bf12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 0x561889bf12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 -- 172.21.15.37:0/4107414558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561889bd49b0 con 0x561889bce620 2024-04-25T21:24:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 0x561889bf12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e88006010 tx=0x7f2e88009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40969163b825e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e82ffd640 1 -- 172.21.15.37:0/4107414558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88009620 con 0x561889bce620 2024-04-25T21:24:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e82ffd640 1 -- 172.21.15.37:0/4107414558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8800f730 con 0x561889bce620 2024-04-25T21:24:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.124+0000 7f2e82ffd640 1 -- 172.21.15.37:0/4107414558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88004580 con 0x561889bce620 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- 172.21.15.37:0/4107414558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 msgr2=0x561889bd8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2e88006010 tx=0x7f2e88009970 comp rx=0 tx=0).stop 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- 172.21.15.37:0/4107414558 shutdown_connections 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bf1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bcef40 0x561889bf12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/4107414558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bce620 0x561889bd8ca0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- 172.21.15.37:0/4107414558 >> 172.21.15.37:0/4107414558 conn(0x561889bcab30 msgr2=0x561889bd6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- 172.21.15.37:0/4107414558 shutdown_connections 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- 172.21.15.37:0/4107414558 wait complete. 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.125+0000 7f2ea21f3740 1 -- start start 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 Processor -- start 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889be1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bdf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b11b0 con 0x561889bcef40 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981b0ae0 con 0x561889bce620 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2e981add40 con 0x561889bd9800 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2ea21f3740 10 client.0 _ll_register_callbacks cb 0x7f2e8800ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33394/0 (socket says 172.21.15.37:33394) 2024-04-25T21:24:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 -- 172.21.15.37:0/471037711 learned_addr learned my addr 172.21.15.37:0/471037711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889be1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 -- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 msgr2=0x561889bdf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bdf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bdf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 -- 172.21.15.37:0/471037711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 msgr2=0x561889be1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889be1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9d27d640 1 -- 172.21.15.37:0/471037711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561889bdf7f0 con 0x561889bcef40 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.126+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e9d27d640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e50 tx=0x7f2e9401f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9a4adf76ec61f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94003d20 con 0x561889bcef40 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8557+0+0 (secure 0 0 0) 0x7f2e94010430 con 0x561889bcef40 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e8800f730 con 0x561889bcef40 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e9402f050 con 0x561889bcef40 2024-04-25T21:24:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 10 client.17181 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561889c4f8f0 con 0x561889bcef40 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8800f730 con 0x561889bcef40 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f2e940274e0 con 0x561889bcef40 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 10 client.17181 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 20 client.17181 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889c50050 0x561889c52470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 4 client.17181 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.127+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561889b3b200 con 0x561889c50050 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.128+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889c50050 0x561889c52470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.128+0000 7f2e9ca7c640 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889c50050 0x561889c52470 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8f5a8726c4441d server_cookie=ae7cbc8544894d9d in_seq=0 out_seq=0 2024-04-25T21:24:04.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.128+0000 7f2e80ff9640 10 client.17181 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2e80ff9640 1 -- 172.21.15.37:0/471037711 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561889b3b200 con 0x561889c50050 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2e80ff9640 10 client.17181 handle_command_reply: tid=0 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 client.17181 shutdown 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 msgr2=0x561889bdecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009e50 tx=0x7f2e9401f770 comp rx=0 tx=0).stop 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 shutdown_connections 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x561889c50050 0x561889c52470 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561889bd9800 0x561889bdf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561889bcef40 0x561889bdecf0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2e80ff9640 0 client.17181 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 --2- 172.21.15.37:0/471037711 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561889bce620 0x561889be1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.143+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 >> 172.21.15.37:0/471037711 conn(0x561889bcab30 msgr2=0x561889bc9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 shutdown_connections 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2ea21f3740 1 -- 172.21.15.37:0/471037711 wait complete. 2024-04-25T21:24:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2ea21f3740 20 client.17181 trim_cache size 0 max 16384 2024-04-25T21:24:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:24:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:24:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:24:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:24:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 msgr2=0x7f2e840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 0x7f2e840a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e900bd840 tx=0x7f2e9002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 msgr2=0x7f2e9819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e90005cc0 tx=0x7f2e90031700 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 shutdown_connections 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e9819a670 0x7f2e9819aaf0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e981061b0 0x7f2e981a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2e840a3dc0 0x7f2e840a6280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 --2- 172.21.15.37:0/699087908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e981010e0 0x7f2e981a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.144+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 >> 172.21.15.37:0/699087908 conn(0x7f2e980771d0 msgr2=0x7f2e9810cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.145+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 shutdown_connections 2024-04-25T21:24:04.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:04.145+0000 7f2e9f4e1640 1 -- 172.21.15.37:0/699087908 wait complete. 2024-04-25T21:24:04.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:03 smithi098 ceph-mon[32569]: pgmap v489: 97 pgs: 97 active+clean; 2.2 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-25T21:24:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:05 smithi037 ceph-mon[29195]: pgmap v490: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:05.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:05 smithi037 ceph-mon[29195]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T21:24:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:05 smithi177 ceph-mon[33043]: pgmap v490: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:05 smithi177 ceph-mon[33043]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T21:24:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:05 smithi098 ceph-mon[32569]: pgmap v490: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:05 smithi098 ceph-mon[32569]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T21:24:07.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:07 smithi037 ceph-mon[29195]: pgmap v492: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:24:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:07 smithi177 ceph-mon[33043]: pgmap v492: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:24:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:07 smithi098 ceph-mon[32569]: pgmap v492: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 130 KiB/s wr, 48 op/s 2024-04-25T21:24:09.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:09 smithi037 ceph-mon[29195]: pgmap v493: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:09 smithi177 ceph-mon[33043]: pgmap v493: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:09 smithi098 ceph-mon[32569]: pgmap v493: 97 pgs: 97 active+clean; 2.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:11.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:11 smithi037 ceph-mon[29195]: pgmap v494: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 120 KiB/s wr, 45 op/s 2024-04-25T21:24:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:11 smithi177 ceph-mon[33043]: pgmap v494: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 120 KiB/s wr, 45 op/s 2024-04-25T21:24:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:11 smithi098 ceph-mon[32569]: pgmap v494: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 120 KiB/s wr, 45 op/s 2024-04-25T21:24:12.901 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.2 2024-04-25T21:24:12.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:24:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:24:13.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:13 smithi037 ceph-mon[29195]: pgmap v495: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:13 smithi177 ceph-mon[33043]: pgmap v495: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:13 smithi098 ceph-mon[32569]: pgmap v495: 97 pgs: 97 active+clean; 2.7 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 -- 172.21.15.37:0/943800553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 msgr2=0x7fe4e810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 --2- 172.21.15.37:0/943800553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e810f930 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000b0a0 tx=0x7fe4e000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 -- 172.21.15.37:0/943800553 shutdown_connections 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 --2- 172.21.15.37:0/943800553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e810f930 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 --2- 172.21.15.37:0/943800553 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 0x7fe4e8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 --2- 172.21.15.37:0/943800553 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 -- 172.21.15.37:0/943800553 >> 172.21.15.37:0/943800553 conn(0x7fe4e80789e0 msgr2=0x7fe4e8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:15.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 -- 172.21.15.37:0/943800553 shutdown_connections 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.036+0000 7fe4ef632700 1 -- 172.21.15.37:0/943800553 wait complete. 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 Processor -- start 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 -- start start 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 0x7fe4e819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8112ca0 con 0x7fe4e806bbb0 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8112b20 con 0x7fe4e806b1f0 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ef632700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8112e20 con 0x7fe4e8108a60 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38912/0 (socket says 172.21.15.37:38912) 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 -- 172.21.15.37:0/770819979 learned_addr learned my addr 172.21.15.37:0/770819979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ed3ce700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:15.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4edbcf700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 0x7fe4e819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 -- 172.21.15.37:0/770819979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 msgr2=0x7fe4e819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 0x7fe4e819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 -- 172.21.15.37:0/770819979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 msgr2=0x7fe4e81990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e81990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ecbcd700 1 -- 172.21.15.37:0/770819979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e819e0e0 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.037+0000 7fe4ed3ce700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4ecbcd700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9a0 tx=0x7fe4d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800ca10 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d800cbb0 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e819e3d0 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8018680 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.038+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e81a5c50 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.039+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e8069730 con 0x7fe4e806bbb0 2024-04-25T21:24:15.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.040+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4d8010430 con 0x7fe4e806bbb0 2024-04-25T21:24:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.040+0000 7fe4de7fc700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 0x7fe4d40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.041+0000 7fe4ed3ce700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 0x7fe4d40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.041+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fe4d80d0970 con 0x7fe4e806bbb0 2024-04-25T21:24:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.041+0000 7fe4ed3ce700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 0x7fe4d40a6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40103d0 tx=0x7fe4e400b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:15.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.045+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4d809d010 con 0x7fe4e806bbb0 2024-04-25T21:24:15.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.301+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4e819a430 con 0x7fe4e806bbb0 2024-04-25T21:24:15.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.302+0000 7fe4de7fc700 1 -- 172.21.15.37:0/770819979 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fe4d809c910 con 0x7fe4e806bbb0 2024-04-25T21:24:15.301 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:24:15.301 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 msgr2=0x7fe4d40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 0x7fe4d40a6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40103d0 tx=0x7fe4e400b3c0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 msgr2=0x7fe4e8199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9a0 tx=0x7fe4d800ecb0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 shutdown_connections 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fe4d40a3db0 0x7fe4d40a6270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4e8108a60 0x7fe4e819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e806b1f0 0x7fe4e81990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 --2- 172.21.15.37:0/770819979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4e806bbb0 0x7fe4e8199610 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 >> 172.21.15.37:0/770819979 conn(0x7fe4e80789e0 msgr2=0x7fe4e80ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 shutdown_connections 2024-04-25T21:24:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:15.304+0000 7fe4ef632700 1 -- 172.21.15.37:0/770819979 wait complete. 2024-04-25T21:24:15.304 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:24:15.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:15 smithi098 ceph-mon[32569]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:24:15.969 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:15 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/770819979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:15 smithi037 ceph-mon[29195]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:24:15.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:15 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/770819979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:15 smithi177 ceph-mon[33043]: pgmap v496: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:24:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:15 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/770819979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:16.152 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 84.2 --await-for 42.0' 2024-04-25T21:24:16.152 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 84.2 --await-for 42.0 2024-04-25T21:24:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 -- 172.21.15.37:0/3519711536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9870102b70 msgr2=0x7f9870106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 --2- 172.21.15.37:0/3519711536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9870102b70 0x7f9870106bb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a3f0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 -- 172.21.15.37:0/3519711536 shutdown_connections 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 --2- 172.21.15.37:0/3519711536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9870107290 0x7f987010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 --2- 172.21.15.37:0/3519711536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9870102b70 0x7f9870106bb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 --2- 172.21.15.37:0/3519711536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f98701021c0 0x7f98701025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 -- 172.21.15.37:0/3519711536 >> 172.21.15.37:0/3519711536 conn(0x7f98700fda20 msgr2=0x7f98700ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 -- 172.21.15.37:0/3519711536 shutdown_connections 2024-04-25T21:24:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.238+0000 7f987508a640 1 -- 172.21.15.37:0/3519711536 wait complete. 2024-04-25T21:24:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 Processor -- start 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 -- start start 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98701021c0 0x7f98701986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9870107290 0x7f9870198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98701121b0 con 0x7f987019cfb0 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112030 con 0x7f98701021c0 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f987508a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112330 con 0x7f9870107290 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38936/0 (socket says 172.21.15.37:38936) 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 -- 172.21.15.37:0/221556190 learned_addr learned my addr 172.21.15.37:0/221556190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986ed9d640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98701021c0 0x7f98701986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f9867fff640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9870107290 0x7f9870198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 -- 172.21.15.37:0/221556190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9870107290 msgr2=0x7f9870198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9870107290 0x7f9870198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 -- 172.21.15.37:0/221556190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98701021c0 msgr2=0x7f98701986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98701021c0 0x7f98701986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.239+0000 7f986f59e640 1 -- 172.21.15.37:0/221556190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987019daa0 con 0x7f987019cfb0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f986f59e640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f986800cad0 tx=0x7f986800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f986cd99640 1 -- 172.21.15.37:0/221556190 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986800ae20 con 0x7f987019cfb0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f986cd99640 1 -- 172.21.15.37:0/221556190 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9868005020 con 0x7f987019cfb0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987019dd90 con 0x7f987019cfb0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f986cd99640 1 -- 172.21.15.37:0/221556190 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986800f6b0 con 0x7f987019cfb0 2024-04-25T21:24:17.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.240+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701a5550 con 0x7f987019cfb0 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f986cd99640 1 -- 172.21.15.37:0/221556190 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98680051d0 con 0x7f987019cfb0 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f9877da2740 1 Processor -- start 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f9877da2740 1 -- start start 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f9877da2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f9877da2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.242+0000 7f9877da2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc34000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9877da2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55775cb87970 con 0x55775cc22500 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9877da2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55775cb244a0 con 0x55775cc31c10 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9877da2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55775cb83930 con 0x55775cc22e20 2024-04-25T21:24:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38946/0 (socket says 172.21.15.37:38946) 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 -- 172.21.15.37:0/3340727874 learned_addr learned my addr 172.21.15.37:0/3340727874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f986cd99640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 0x7f985c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f986ed9d640 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f986f59e640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 0x7f985c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 -- 172.21.15.37:0/3340727874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 msgr2=0x55775cc316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc316d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 -- 172.21.15.37:0/3340727874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 msgr2=0x55775cc34000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 -- 172.21.15.37:0/3340727874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775c89a210 con 0x55775cc22500 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f986cd99640 1 -- 172.21.15.37:0/221556190 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f98680d0870 con 0x7f987019cfb0 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f9867fff640 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f987006b5f0 tx=0x7f9858005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3be9cbc7f6bf9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f986f59e640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 0x7f985c0a6300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f986800caa0 tx=0x7f9868087040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f98657fa640 1 -- 172.21.15.37:0/3340727874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858035050 con 0x55775cc22500 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f98657fa640 1 -- 172.21.15.37:0/3340727874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98580316a0 con 0x55775cc22500 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.243+0000 7f98657fa640 1 -- 172.21.15.37:0/3340727874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580046c0 con 0x55775cc22500 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 -- 172.21.15.37:0/3340727874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 msgr2=0x55775cc228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f987006b5f0 tx=0x7f9858005e10 comp rx=0 tx=0).stop 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 -- 172.21.15.37:0/3340727874 shutdown_connections 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc34000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 --2- 172.21.15.37:0/3340727874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc228e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 -- 172.21.15.37:0/3340727874 >> 172.21.15.37:0/3340727874 conn(0x55775cc1eb30 msgr2=0x55775cc1ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 -- 172.21.15.37:0/3340727874 shutdown_connections 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.244+0000 7f9877da2740 1 -- 172.21.15.37:0/3340727874 wait complete. 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 -- start start 2024-04-25T21:24:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 Processor -- start 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc2e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55775cc347e0 con 0x55775cc22500 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55775cb220b0 con 0x55775cc31c10 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55775cc34960 con 0x55775cc22e20 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9877da2740 10 client.0 _ll_register_callbacks cb 0x55775cb0fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38954/0 (socket says 172.21.15.37:38954) 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 -- 172.21.15.37:0/3387176742 learned_addr learned my addr 172.21.15.37:0/3387176742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f986ed9d640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc2e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 -- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 msgr2=0x55775cc2e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc2e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 -- 172.21.15.37:0/3387176742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 msgr2=0x55775cc2ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc2ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.245+0000 7f9867fff640 1 -- 172.21.15.37:0/3387176742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775c966620 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9867fff640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9858005ec0 tx=0x7f9858005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb36e969381876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858035040 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f9858004030 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98580316a0 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9858004230 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 10 client.17199 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55775cc40730 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98580316a0 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9858003710 con 0x55775cc22500 2024-04-25T21:24:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 10 client.17199 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 20 client.17199 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc41a80 0x55775cc43e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 4 client.17199 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.246+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55775cb95390 con 0x55775cc41a80 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.247+0000 7f986ed9d640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc41a80 0x55775cc43e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.247+0000 7f986ed9d640 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc41a80 0x55775cc43e70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe87401bc8eab350 server_cookie=202ecb7cbfa82bcc in_seq=0 out_seq=0 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.247+0000 7f98567fc640 10 client.17199 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f98567fc640 1 -- 172.21.15.37:0/3387176742 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55775cb95390 con 0x55775cc41a80 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f98567fc640 10 client.17199 handle_command_reply: tid=0 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 client.17199 shutdown 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 msgr2=0x55775cc31510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f9858005ec0 tx=0x7f9858005a20 comp rx=0 tx=0).stop 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 shutdown_connections 2024-04-25T21:24:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc41a80 0x55775cc43e70 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc31c10 0x55775cc2ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc2e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 --2- 172.21.15.37:0/3387176742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31510 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f98567fc640 0 client.17199 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 >> 172.21.15.37:0/3387176742 conn(0x55775cc1eb30 msgr2=0x55775cc249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 shutdown_connections 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 1 -- 172.21.15.37:0/3387176742 wait complete. 2024-04-25T21:24:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.248+0000 7f9877da2740 20 client.17199 trim_cache size 0 max 16384 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 Processor -- start 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 -- start start 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc1eb30 0x55775cc29f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc49490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98701ab8b0 con 0x55775cc22500 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98701ad3f0 con 0x55775cc1eb30 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f9877da2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98701abf80 con 0x55775cc22e20 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f986ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f986ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38958/0 (socket says 172.21.15.37:38958) 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.263+0000 7f986ed9d640 1 -- 172.21.15.37:0/340965414 learned_addr learned my addr 172.21.15.37:0/340965414 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f9867fff640 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc49490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f9867fff640 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc1eb30 0x55775cc29f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 -- 172.21.15.37:0/340965414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 msgr2=0x55775cc49490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc49490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 -- 172.21.15.37:0/340965414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc1eb30 msgr2=0x55775cc29f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc1eb30 0x55775cc29f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 -- 172.21.15.37:0/340965414 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55775cc22c50 con 0x55775cc22500 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f986ed9d640 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x55775cc2e060 tx=0x7f9860014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cad8919777c8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f9864ff9640 1 -- 172.21.15.37:0/340965414 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860014190 con 0x55775cc22500 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f9864ff9640 1 -- 172.21.15.37:0/340965414 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860022020 con 0x55775cc22500 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.264+0000 7f9864ff9640 1 -- 172.21.15.37:0/340965414 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860003e60 con 0x55775cc22500 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- 172.21.15.37:0/340965414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 msgr2=0x55775cc31040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x55775cc2e060 tx=0x7f9860014900 comp rx=0 tx=0).stop 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- 172.21.15.37:0/340965414 shutdown_connections 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc22e20 0x55775cc49490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22500 0x55775cc31040 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- 172.21.15.37:0/340965414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc1eb30 0x55775cc29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- 172.21.15.37:0/340965414 >> 172.21.15.37:0/340965414 conn(0x55775cc2eea0 msgr2=0x55775cc23fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- 172.21.15.37:0/340965414 shutdown_connections 2024-04-25T21:24:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- 172.21.15.37:0/340965414 wait complete. 2024-04-25T21:24:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- start start 2024-04-25T21:24:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 Processor -- start 2024-04-25T21:24:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc1eb30 0x55775cc355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc22500 0x55775cc35af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98701acd20 con 0x55775cc22e20 2024-04-25T21:24:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98701ac650 con 0x55775cc22500 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.265+0000 7f9877da2740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f98701a98b0 con 0x55775cc1eb30 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9877da2740 10 client.0 _ll_register_callbacks cb 0x7f9858005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38972/0 (socket says 172.21.15.37:38972) 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 -- 172.21.15.37:0/4228826874 learned_addr learned my addr 172.21.15.37:0/4228826874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc1eb30 0x55775cc355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f986ed9d640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc22500 0x55775cc35af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 -- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc1eb30 msgr2=0x55775cc355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc1eb30 0x55775cc355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 -- 172.21.15.37:0/4228826874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc22500 msgr2=0x55775cc35af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc22500 0x55775cc35af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 -- 172.21.15.37:0/4228826874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55775cca6d00 con 0x55775cc22e20 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9867fff640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9858005ab0 tx=0x7f9858005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a486752adcd553bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f98567fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985805c070 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9860022020 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f9858003d00 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9858035040 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9877da2740 10 client.17205 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.266+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55775cca7fb0 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9860022020 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9858056550 con 0x55775cc22e20 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 10 client.17205 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 20 client.17205 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc31c10 0x55775ccaa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 4 client.17205 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55775cb8f200 con 0x55775cc31c10 2024-04-25T21:24:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f986ed9d640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc31c10 0x55775ccaa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f986ed9d640 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc31c10 0x55775ccaa5f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c431f768c80f5771 server_cookie=976a96574fc0a2a7 in_seq=0 out_seq=0 2024-04-25T21:24:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.267+0000 7f98567fc640 10 client.17205 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.281+0000 7f98567fc640 1 -- 172.21.15.37:0/4228826874 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55775cb8f200 con 0x55775cc31c10 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.281+0000 7f98567fc640 10 client.17205 handle_command_reply: tid=0 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.281+0000 7f9877da2740 1 client.17205 shutdown 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 msgr2=0x55775cc36030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9858005ab0 tx=0x7f9858005750 comp rx=0 tx=0).stop 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 shutdown_connections 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55775cc31c10 0x55775ccaa5f0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55775cc22e20 0x55775cc36030 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55775cc22500 0x55775cc35af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 --2- 172.21.15.37:0/4228826874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55775cc1eb30 0x55775cc355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f98567fc640 0 client.17205 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 >> 172.21.15.37:0/4228826874 conn(0x55775cc2eea0 msgr2=0x55775cc29d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 shutdown_connections 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 1 -- 172.21.15.37:0/4228826874 wait complete. 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f9877da2740 20 client.17205 trim_cache size 0 max 16384 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:24:17.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:24:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 msgr2=0x7f985c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 0x7f985c0a6300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f986800caa0 tx=0x7f9868087040 comp rx=0 tx=0).stop 2024-04-25T21:24:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 msgr2=0x7f987019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f986800cad0 tx=0x7f986800cf90 comp rx=0 tx=0).stop 2024-04-25T21:24:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 shutdown_connections 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f987019cfb0 0x7f987019d460 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f985c0a3e40 0x7f985c0a6300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9870107290 0x7f9870198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.282+0000 7f987508a640 1 --2- 172.21.15.37:0/221556190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98701021c0 0x7f98701986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.283+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 >> 172.21.15.37:0/221556190 conn(0x7f98700fda20 msgr2=0x7f98700fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.283+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 shutdown_connections 2024-04-25T21:24:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:17.283+0000 7f987508a640 1 -- 172.21.15.37:0/221556190 wait complete. 2024-04-25T21:24:17.889 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:17 smithi037 ceph-mon[29195]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-04-25T21:24:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:17 smithi177 ceph-mon[33043]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-04-25T21:24:18.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:17 smithi098 ceph-mon[32569]: pgmap v497: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.269+0000 7f9427885700 1 -- 172.21.15.37:0/1478859231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201019f0 msgr2=0x7f9420101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.269+0000 7f9427885700 1 --2- 172.21.15.37:0/1478859231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201019f0 0x7f9420101e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f9414009990 tx=0x7f9414009ca0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- 172.21.15.37:0/1478859231 shutdown_connections 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- 172.21.15.37:0/1478859231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201023b0 0x7f942010e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- 172.21.15.37:0/1478859231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 0x7f9420107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- 172.21.15.37:0/1478859231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201019f0 0x7f9420101e70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- 172.21.15.37:0/1478859231 >> 172.21.15.37:0/1478859231 conn(0x7f9420078990 msgr2=0x7f9420078da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- 172.21.15.37:0/1478859231 shutdown_connections 2024-04-25T21:24:18.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- 172.21.15.37:0/1478859231 wait complete. 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 Processor -- start 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- start start 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 0x7f94201035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 0x7f9420104040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111c30 con 0x7f94201023b0 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111ab0 con 0x7f94201079f0 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.270+0000 7f9427885700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111db0 con 0x7f94201019f0 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38990/0 (socket says 172.21.15.37:38990) 2024-04-25T21:24:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 -- 172.21.15.37:0/353907140 learned_addr learned my addr 172.21.15.37:0/353907140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9425e22700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 0x7f9420104040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9425621700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 0x7f94201035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 -- 172.21.15.37:0/353907140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 msgr2=0x7f94201035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 0x7f94201035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 -- 172.21.15.37:0/353907140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 msgr2=0x7f9420104040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 0x7f9420104040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 -- 172.21.15.37:0/353907140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94201a7660 con 0x7f94201023b0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9425621700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 0x7f94201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9424e20700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c00 tx=0x7f94140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f941e7fc700 1 -- 172.21.15.37:0/353907140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414027070 con 0x7f94201023b0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f941e7fc700 1 -- 172.21.15.37:0/353907140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9414003f20 con 0x7f94201023b0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94201a78f0 con 0x7f94201023b0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.271+0000 7f941e7fc700 1 -- 172.21.15.37:0/353907140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414022cf0 con 0x7f94201023b0 2024-04-25T21:24:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.272+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94201a7da0 con 0x7f94201023b0 2024-04-25T21:24:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.274+0000 7f941e7fc700 1 -- 172.21.15.37:0/353907140 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f941402a070 con 0x7f94201023b0 2024-04-25T21:24:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.274+0000 7f941e7fc700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 0x7f94080a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.275+0000 7f9425621700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 0x7f94080a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.275+0000 7f941e7fc700 1 -- 172.21.15.37:0/353907140 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f94140d1590 con 0x7f94201023b0 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.275+0000 7f9425621700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 0x7f94080a6390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94180099d0 tx=0x7f9418008040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.275+0000 7f943210eb80 1 Processor -- start 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.275+0000 7f943210eb80 1 -- start start 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374796cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x5637479779c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5637478bfe20 con 0x563747977f00 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563747866ac0 con 0x56374796c6e0 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f943210eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5637478bbde0 con 0x56374796d000 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38996/0 (socket says 172.21.15.37:38996) 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 -- 172.21.15.37:0/3147018142 learned_addr learned my addr 172.21.15.37:0/3147018142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425621700 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374796cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9424e20700 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x5637479779c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 -- 172.21.15.37:0/3147018142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 msgr2=0x5637479779c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x5637479779c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 -- 172.21.15.37:0/3147018142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 msgr2=0x56374796cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374796cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 -- 172.21.15.37:0/3147018142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374782a8d0 con 0x563747977f00 2024-04-25T21:24:18.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f9425e22700 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f94100098f0 tx=0x7f9410009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e1b0d526bbcc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.276+0000 7f94077fe700 1 -- 172.21.15.37:0/3147018142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004bb0 con 0x563747977f00 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f94077fe700 1 -- 172.21.15.37:0/3147018142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9410004d50 con 0x563747977f00 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f94077fe700 1 -- 172.21.15.37:0/3147018142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410007da0 con 0x563747977f00 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 -- 172.21.15.37:0/3147018142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 msgr2=0x56374797a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f94100098f0 tx=0x7f9410009c00 comp rx=0 tx=0).stop 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 -- 172.21.15.37:0/3147018142 shutdown_connections 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x56374797a2f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374796cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 --2- 172.21.15.37:0/3147018142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x5637479779c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 -- 172.21.15.37:0/3147018142 >> 172.21.15.37:0/3147018142 conn(0x563747899a20 msgr2=0x563747968ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 -- 172.21.15.37:0/3147018142 shutdown_connections 2024-04-25T21:24:18.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.277+0000 7f943210eb80 1 -- 172.21.15.37:0/3147018142 wait complete. 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 -- start start 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 Processor -- start 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x563747977590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x563747974900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56374797a990 con 0x563747977f00 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x563747879740 con 0x56374796c6e0 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56374797ab10 con 0x56374796d000 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f943210eb80 10 client.0 _ll_register_callbacks cb 0x563747858ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f9425e22700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f9424e20700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x563747974900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f9425e22700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39012/0 (socket says 172.21.15.37:39012) 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f9425e22700 1 -- 172.21.15.37:0/2181755 learned_addr learned my addr 172.21.15.37:0/2181755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.278+0000 7f9425621700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x563747977590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 -- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 msgr2=0x563747974900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x563747974900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 -- 172.21.15.37:0/2181755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 msgr2=0x563747977590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x563747977590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 -- 172.21.15.37:0/2181755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374780da60 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9425e22700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9410007980 tx=0x7f9410004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6166afc5b3a28bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9404ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410022070 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f94100079e0 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9410004d50 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f941002e050 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f943210eb80 10 client.17217 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.279+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563747985200 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9410004d50 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f941001e410 con 0x563747977f00 2024-04-25T21:24:18.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 10 client.17217 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 10 client.17217 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 20 client.17217 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563747987730 0x563747989b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 4 client.17217 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637478cd610 con 0x563747987730 2024-04-25T21:24:18.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.280+0000 7f9425621700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563747987730 0x563747989b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f9425621700 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563747987730 0x563747989b20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9bba1a2f3f691f2 server_cookie=83d4cebe81519156 in_seq=0 out_seq=0 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f9404ff9700 10 client.17217 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f9404ff9700 1 -- 172.21.15.37:0/2181755 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637478cd610 con 0x563747987730 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f9404ff9700 10 client.17217 handle_command_reply: tid=0 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f943210eb80 1 client.17217 shutdown 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 msgr2=0x563747974e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9410007980 tx=0x7f9410004550 comp rx=0 tx=0).stop 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 shutdown_connections 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.281+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x563747987730 0x563747989b20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563747977f00 0x563747974e40 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x563747977590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 --2- 172.21.15.37:0/2181755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56374796d000 0x563747974900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 >> 172.21.15.37:0/2181755 conn(0x563747899a20 msgr2=0x56374796f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f9404ff9700 0 client.17217 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 shutdown_connections 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 1 -- 172.21.15.37:0/2181755 wait complete. 2024-04-25T21:24:18.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.282+0000 7f943210eb80 20 client.17217 trim_cache size 0 max 16384 2024-04-25T21:24:18.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.301+0000 7f943210eb80 1 Processor -- start 2024-04-25T21:24:18.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.301+0000 7f943210eb80 1 -- start start 2024-04-25T21:24:18.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x563747971b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94201af190 con 0x56374796d000 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94201b0cd0 con 0x56374796c6e0 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f943210eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94201af860 con 0x563747977f00 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39028/0 (socket says 172.21.15.37:39028) 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 -- 172.21.15.37:0/4123649654 learned_addr learned my addr 172.21.15.37:0/4123649654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9425621700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9425e22700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x563747971b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 -- 172.21.15.37:0/4123649654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 msgr2=0x563747971b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x563747971b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 -- 172.21.15.37:0/4123649654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 msgr2=0x56374797fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 -- 172.21.15.37:0/4123649654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374796e140 con 0x56374796d000 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9425621700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9425e22700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x563747971b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9424e20700 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f94140cece0 tx=0x7f9414088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4170a7e4107b7da1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9406ffd700 1 -- 172.21.15.37:0/4123649654 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140a6140 con 0x56374796d000 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9406ffd700 1 -- 172.21.15.37:0/4123649654 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94140cef00 con 0x56374796d000 2024-04-25T21:24:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.302+0000 7f9406ffd700 1 -- 172.21.15.37:0/4123649654 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140a5f10 con 0x56374796d000 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 -- 172.21.15.37:0/4123649654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 msgr2=0x563747980300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f94140cece0 tx=0x7f9414088040 comp rx=0 tx=0).stop 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 -- 172.21.15.37:0/4123649654 shutdown_connections 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x563747971b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 --2- 172.21.15.37:0/4123649654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x563747980300 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 -- 172.21.15.37:0/4123649654 >> 172.21.15.37:0/4123649654 conn(0x563747899a20 msgr2=0x5637479828b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 -- 172.21.15.37:0/4123649654 shutdown_connections 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.303+0000 7f943210eb80 1 -- 172.21.15.37:0/4123649654 wait complete. 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 -- start start 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 Processor -- start 2024-04-25T21:24:18.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x5637479eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f94201b0600 con 0x56374796d000 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94201aff30 con 0x56374796c6e0 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f94201ad150 con 0x563747977f00 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f943210eb80 10 client.0 _ll_register_callbacks cb 0x563747858ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39030/0 (socket says 172.21.15.37:39030) 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 -- 172.21.15.37:0/206217402 learned_addr learned my addr 172.21.15.37:0/206217402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9425621700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 -- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 msgr2=0x5637479eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9425e22700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x5637479eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x5637479eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 -- 172.21.15.37:0/206217402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 msgr2=0x56374797bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9424e20700 1 -- 172.21.15.37:0/206217402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637479ecf60 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9425e22700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x5637479eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.304+0000 7f9425621700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9424e20700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9414005cb0 tx=0x7f941409da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b52d86f2ae0493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94140a6140 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f94140a5ec0 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94140cef00 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94140bc2e0 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 10 client.17223 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637479ee120 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94140cef00 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f94140d1160 con 0x56374796d000 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 10 client.17223 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 10 client.17223 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:24:18.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 20 client.17223 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637479ee8c0 0x5637479f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 4 client.17223 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.2, "await_for": 42.0}] 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.305+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.2, "await_for": 42.0}) -- 0x5637478c76b0 con 0x5637479ee8c0 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.306+0000 7f9425621700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637479ee8c0 0x5637479f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.306+0000 7f9404ff9700 10 client.17223 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:18.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.306+0000 7f9425621700 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637479ee8c0 0x5637479f0ce0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36ff6f2c1bba626e server_cookie=aa9861c44a164ad5 in_seq=0 out_seq=0 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f9404ff9700 1 -- 172.21.15.37:0/206217402 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5637478c76b0 con 0x5637479ee8c0 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f9404ff9700 10 client.17223 handle_command_reply: tid=0 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 client.17223 shutdown 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 msgr2=0x56374797c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9414005cb0 tx=0x7f941409da80 comp rx=0 tx=0).stop 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 shutdown_connections 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5637479ee8c0 0x5637479f0ce0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563747977f00 0x5637479eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56374796c6e0 0x56374797bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f9404ff9700 0 client.17223 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 --2- 172.21.15.37:0/206217402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374796d000 0x56374797c330 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 >> 172.21.15.37:0/206217402 conn(0x563747899a20 msgr2=0x563747983140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 shutdown_connections 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 1 -- 172.21.15.37:0/206217402 wait complete. 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.321+0000 7f943210eb80 20 client.17223 trim_cache size 0 max 16384 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:24:18.322 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 42, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "45c365d2": { 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 42, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.2, 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:24:18.323 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:18.325 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 msgr2=0x7f94080a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 0x7f94080a6390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94180099d0 tx=0x7f9418008040 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 msgr2=0x7f9420103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9414000c00 tx=0x7f94140047b0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 shutdown_connections 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f94080a3ed0 0x7f94080a6390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94201079f0 0x7f9420104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f94201019f0 0x7f94201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 --2- 172.21.15.37:0/353907140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94201023b0 0x7f9420103b00 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 >> 172.21.15.37:0/353907140 conn(0x7f9420078990 msgr2=0x7f942006b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:18.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 shutdown_connections 2024-04-25T21:24:18.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:18.322+0000 7f9427885700 1 -- 172.21.15.37:0/353907140 wait complete. 2024-04-25T21:24:19.054 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '45c365d2', quiesce took 6.2 seconds. Will release after: 24.2 2024-04-25T21:24:19.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:19 smithi037 ceph-mon[29195]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:24:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:19 smithi177 ceph-mon[33043]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:24:20.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:19 smithi098 ceph-mon[32569]: pgmap v498: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-25T21:24:21.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:21 smithi037 ceph-mon[29195]: pgmap v499: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:21 smithi177 ceph-mon[33043]: pgmap v499: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:21 smithi098 ceph-mon[32569]: pgmap v499: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:24:23.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:23 smithi037 ceph-mon[29195]: pgmap v500: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:24:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:23 smithi177 ceph-mon[33043]: pgmap v500: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:24:24.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:23 smithi098 ceph-mon[32569]: pgmap v500: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-25T21:24:25.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:25 smithi037 ceph-mon[29195]: pgmap v501: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-25T21:24:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:25 smithi177 ceph-mon[33043]: pgmap v501: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-25T21:24:26.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:25 smithi098 ceph-mon[32569]: pgmap v501: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-25T21:24:27.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:27 smithi037 ceph-mon[29195]: pgmap v502: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:27 smithi177 ceph-mon[33043]: pgmap v502: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:27 smithi098 ceph-mon[32569]: pgmap v502: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:29.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:29 smithi037 ceph-mon[29195]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:29 smithi177 ceph-mon[33043]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:30.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:29 smithi098 ceph-mon[32569]: pgmap v503: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-25T21:24:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 -- 172.21.15.37:0/1304529256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc404101a90 msgr2=0x7fc404105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 --2- 172.21.15.37:0/1304529256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc404101a90 0x7fc404105ad0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f402a410 comp rx=0 tx=0).stop 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 -- 172.21.15.37:0/1304529256 shutdown_connections 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 --2- 172.21.15.37:0/1304529256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 0x7fc40410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 --2- 172.21.15.37:0/1304529256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc404101a90 0x7fc404105ad0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 --2- 172.21.15.37:0/1304529256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc4041010e0 0x7fc4041014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 -- 172.21.15.37:0/1304529256 >> 172.21.15.37:0/1304529256 conn(0x7fc4040771d0 msgr2=0x7fc4040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 -- 172.21.15.37:0/1304529256 shutdown_connections 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.381+0000 7fc40be57640 1 -- 172.21.15.37:0/1304529256 wait complete. 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 Processor -- start 2024-04-25T21:24:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 -- start start 2024-04-25T21:24:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 0x7fc4041a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40419a690 0x7fc40419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041110f0 con 0x7fc4041010e0 2024-04-25T21:24:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc404110f70 con 0x7fc4041061b0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc40be57640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc404111270 con 0x7fc40419a690 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57440/0 (socket says 172.21.15.37:57440) 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc409bf3640 1 -- 172.21.15.37:0/556402391 learned_addr learned my addr 172.21.15.37:0/556402391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.382+0000 7fc4093f2640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 0x7fc4041a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40419a690 0x7fc40419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 -- 172.21.15.37:0/556402391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40419a690 msgr2=0x7fc40419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40419a690 0x7fc40419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 -- 172.21.15.37:0/556402391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 msgr2=0x7fc4041a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 0x7fc4041a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 -- 172.21.15.37:0/556402391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40419b290 con 0x7fc4041010e0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc409bf3640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc40000ea80 tx=0x7fc40000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc3faffd640 1 -- 172.21.15.37:0/556402391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000ce40 con 0x7fc4041010e0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc3faffd640 1 -- 172.21.15.37:0/556402391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc400005020 con 0x7fc4041010e0 2024-04-25T21:24:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc3faffd640 1 -- 172.21.15.37:0/556402391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4000116d0 con 0x7fc4041010e0 2024-04-25T21:24:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4041a7370 con 0x7fc4041010e0 2024-04-25T21:24:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.383+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041a7720 con 0x7fc4041010e0 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.386+0000 7fc3faffd640 1 -- 172.21.15.37:0/556402391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc4000054d0 con 0x7fc4041010e0 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.386+0000 7fc40eb68740 1 Processor -- start 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.386+0000 7fc40eb68740 1 -- start start 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d428ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5643d41f1970 con 0x5643d4297800 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5643d418e4a0 con 0x5643d428c620 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40eb68740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643d41ed930 con 0x5643d428cf40 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc3faffd640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 0x7fc3f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57444/0 (socket says 172.21.15.37:57444) 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 -- 172.21.15.37:0/501862296 learned_addr learned my addr 172.21.15.37:0/501862296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 0x7fc3f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d428ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc3faffd640 1 -- 172.21.15.37:0/556402391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fc4000d0a50 con 0x7fc4041010e0 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc409bf3640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 -- 172.21.15.37:0/501862296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 msgr2=0x5643d42972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 -- 172.21.15.37:0/501862296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 msgr2=0x5643d428ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d428ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc4093f2640 1 -- 172.21.15.37:0/501862296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d3f04210 con 0x5643d4297800 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.387+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d428ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.388+0000 7fc4093f2640 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af71011f8a5a194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.388+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 0x7fc3f00a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00bfd0 tx=0x7fc3fc00b750 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.388+0000 7fc3effff640 1 -- 172.21.15.37:0/501862296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4040070 con 0x5643d4297800 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.388+0000 7fc3effff640 1 -- 172.21.15.37:0/501862296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f4004030 con 0x5643d4297800 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.388+0000 7fc3effff640 1 -- 172.21.15.37:0/501862296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4035040 con 0x5643d4297800 2024-04-25T21:24:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- 172.21.15.37:0/501862296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 msgr2=0x5643d4299bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f4005c00 comp rx=0 tx=0).stop 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- 172.21.15.37:0/501862296 shutdown_connections 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4299bf0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 --2- 172.21.15.37:0/501862296 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d428ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- 172.21.15.37:0/501862296 >> 172.21.15.37:0/501862296 conn(0x5643d4288b30 msgr2=0x5643d4288f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- 172.21.15.37:0/501862296 shutdown_connections 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- 172.21.15.37:0/501862296 wait complete. 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 -- start start 2024-04-25T21:24:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.389+0000 7fc40eb68740 1 Processor -- start 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d4294410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d4294950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5643d429a3d0 con 0x5643d428c620 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5643d418c0b0 con 0x5643d428cf40 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5643d429a550 con 0x5643d4297800 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 10 client.0 _ll_register_callbacks cb 0x5643d4179d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57446/0 (socket says 172.21.15.37:57446) 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 -- 172.21.15.37:0/2289585308 learned_addr learned my addr 172.21.15.37:0/2289585308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc4093f2640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d4294410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d4294950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 -- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 msgr2=0x5643d4294950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d4294950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 -- 172.21.15.37:0/2289585308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 msgr2=0x5643d4294410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d4294410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 -- 172.21.15.37:0/2289585308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d3fd0620 con 0x5643d428c620 2024-04-25T21:24:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d4294950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc409bf3640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c700 tx=0x7fc40009c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123091ee1cb74d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001f050 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fc4000bbbb0 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.390+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f4004030 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc4000a0da0 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 10 client.17235 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643d42a62b0 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f4004030 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc40000f040 con 0x5643d428c620 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 10 client.17235 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 20 client.17235 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d42a77c0 0x5643d42a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 4 client.17235 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d41ff390 con 0x5643d42a77c0 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.391+0000 7fc4093f2640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d42a77c0 0x5643d42a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.392+0000 7fc4093f2640 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d42a77c0 0x5643d42a9bd0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e1da64700aa1eb8 server_cookie=8172d6672eeaff19 in_seq=0 out_seq=0 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.392+0000 7fc3ed7fa640 10 client.17235 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/2289585308 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643d41ff390 con 0x5643d42a77c0 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc3ed7fa640 10 client.17235 handle_command_reply: tid=0 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 client.17235 shutdown 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 msgr2=0x5643d4297120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc40000c700 tx=0x7fc40009c9a0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 shutdown_connections 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d42a77c0 0x5643d42a9bd0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d4294950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d4294410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 --2- 172.21.15.37:0/2289585308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d4297120 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc3ed7fa640 0 client.17235 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 >> 172.21.15.37:0/2289585308 conn(0x5643d4288b30 msgr2=0x5643d428e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 shutdown_connections 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 1 -- 172.21.15.37:0/2289585308 wait complete. 2024-04-25T21:24:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.393+0000 7fc40eb68740 20 client.17235 trim_cache size 0 max 16384 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 Processor -- start 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 -- start start 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d42a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041afc40 con 0x5643d4297800 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041b1780 con 0x5643d428c620 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40eb68740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041b0310 con 0x5643d428cf40 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40a3f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40a3f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57460/0 (socket says 172.21.15.37:57460) 2024-04-25T21:24:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc40a3f4640 1 -- 172.21.15.37:0/22064447 learned_addr learned my addr 172.21.15.37:0/22064447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc409bf3640 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d42a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.408+0000 7fc4093f2640 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 -- 172.21.15.37:0/22064447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 msgr2=0x5643d42948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 -- 172.21.15.37:0/22064447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 msgr2=0x5643d42a3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d42a3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 -- 172.21.15.37:0/22064447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d4292960 con 0x5643d4297800 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc01f770 tx=0x7fc3fc00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16130d8021a702b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc3ef7fe640 1 -- 172.21.15.37:0/22064447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc005220 con 0x5643d4297800 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc3ef7fe640 1 -- 172.21.15.37:0/22064447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc004c50 con 0x5643d4297800 2024-04-25T21:24:30.410 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.409+0000 7fc3ef7fe640 1 -- 172.21.15.37:0/22064447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc012910 con 0x5643d4297800 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- 172.21.15.37:0/22064447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 msgr2=0x5643d4287420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc01f770 tx=0x7fc3fc00b2e0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- 172.21.15.37:0/22064447 shutdown_connections 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d4297800 0x5643d4287420 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d428cf40 0x5643d42948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- 172.21.15.37:0/22064447 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428c620 0x5643d42a3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- 172.21.15.37:0/22064447 >> 172.21.15.37:0/22064447 conn(0x5643d4288b30 msgr2=0x5643d42941e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- 172.21.15.37:0/22064447 shutdown_connections 2024-04-25T21:24:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- 172.21.15.37:0/22064447 wait complete. 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- start start 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 Processor -- start 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d429d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d429dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041b10b0 con 0x5643d428c620 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041b09e0 con 0x5643d428cf40 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.410+0000 7fc40eb68740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041adc40 con 0x5643d4297800 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40eb68740 10 client.0 _ll_register_callbacks cb 0x7fc400098df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc409bf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57470/0 (socket says 172.21.15.37:57470) 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc409bf3640 1 -- 172.21.15.37:0/1683718095 learned_addr learned my addr 172.21.15.37:0/1683718095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d429dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc4093f2640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d429d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 -- 172.21.15.37:0/1683718095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 msgr2=0x5643d429d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d429d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 -- 172.21.15.37:0/1683718095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 msgr2=0x5643d429d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 -- 172.21.15.37:0/1683718095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643d430cd30 con 0x5643d4297800 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc409bf3640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc40a3f4640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d429dbf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc02b040 tx=0x7fc3fc00bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c26fcd336a5be94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.411+0000 7fc3ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc004a70 con 0x5643d4297800 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3fc004c50 con 0x5643d4297800 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fc3fc012690 con 0x5643d4297800 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc3fc040050 con 0x5643d4297800 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 10 client.26575 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5643d430dfc0 con 0x5643d4297800 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3fc004c50 con 0x5643d4297800 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc3fc01c040 con 0x5643d4297800 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 10 client.26575 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 20 client.26575 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d430e690 0x5643d4310ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 4 client.26575 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643d41f9200 con 0x5643d430e690 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.412+0000 7fc409bf3640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d430e690 0x5643d4310ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.413+0000 7fc409bf3640 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d430e690 0x5643d4310ab0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c73f89d11bed4de server_cookie=7a58b00fbfbcda5e in_seq=0 out_seq=0 2024-04-25T21:24:30.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.413+0000 7fc3ed7fa640 10 client.26575 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc3ed7fa640 1 -- 172.21.15.37:0/1683718095 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643d41f9200 con 0x5643d430e690 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc3ed7fa640 10 client.26575 handle_command_reply: tid=0 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 client.26575 shutdown 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 msgr2=0x5643d429dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d429dbf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc02b040 tx=0x7fc3fc00bf30 comp rx=0 tx=0).stop 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 shutdown_connections 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5643d430e690 0x5643d4310ab0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5643d4297800 0x5643d429dbf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc3ed7fa640 0 client.26575 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5643d428cf40 0x5643d429d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 --2- 172.21.15.37:0/1683718095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643d428c620 0x5643d429d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 >> 172.21.15.37:0/1683718095 conn(0x5643d4288b30 msgr2=0x5643d42a2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 shutdown_connections 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 1 -- 172.21.15.37:0/1683718095 wait complete. 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.427+0000 7fc40eb68740 20 client.26575 trim_cache size 0 max 16384 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:24:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:24:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 msgr2=0x7fc3f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 0x7fc3f00a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00bfd0 tx=0x7fc3fc00b750 comp rx=0 tx=0).stop 2024-04-25T21:24:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 msgr2=0x7fc4041a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc40000ea80 tx=0x7fc40000ef40 comp rx=0 tx=0).stop 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 shutdown_connections 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fc3f00a3dc0 0x7fc3f00a6280 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc40419a690 0x7fc40419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4041061b0 0x7fc4041a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 --2- 172.21.15.37:0/556402391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4041010e0 0x7fc4041a05d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 >> 172.21.15.37:0/556402391 conn(0x7fc4040771d0 msgr2=0x7fc40406a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 shutdown_connections 2024-04-25T21:24:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:30.428+0000 7fc40be57640 1 -- 172.21.15.37:0/556402391 wait complete. 2024-04-25T21:24:31.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:31 smithi037 ceph-mon[29195]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:24:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:31 smithi177 ceph-mon[33043]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:24:32.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:31 smithi098 ceph-mon[32569]: pgmap v504: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-25T21:24:33.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:33 smithi037 ceph-mon[29195]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-25T21:24:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:33 smithi177 ceph-mon[33043]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-25T21:24:34.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:33 smithi098 ceph-mon[32569]: pgmap v505: 97 pgs: 97 active+clean; 3.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-25T21:24:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:35 smithi177 ceph-mon[33043]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 15 op/s 2024-04-25T21:24:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:35 smithi098 ceph-mon[32569]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 15 op/s 2024-04-25T21:24:36.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:35 smithi037 ceph-mon[29195]: pgmap v506: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 15 op/s 2024-04-25T21:24:38.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:37 smithi177 ceph-mon[33043]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:38.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:37 smithi098 ceph-mon[32569]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:38.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:37 smithi037 ceph-mon[29195]: pgmap v507: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:40.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:39 smithi177 ceph-mon[33043]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:40.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:39 smithi098 ceph-mon[32569]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:40.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:39 smithi037 ceph-mon[29195]: pgmap v508: 97 pgs: 97 active+clean; 3.1 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 KiB/s wr, 11 op/s 2024-04-25T21:24:42.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:41 smithi177 ceph-mon[33043]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T21:24:42.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:41 smithi098 ceph-mon[32569]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T21:24:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:41 smithi037 ceph-mon[29195]: pgmap v509: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T21:24:43.256 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 -- 172.21.15.37:0/862308653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d6810c520 msgr2=0x7f1d6810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/862308653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d6810c520 0x7f1d6810e9c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009ca0 tx=0x7f1d6002a9e0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 -- 172.21.15.37:0/862308653 shutdown_connections 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/862308653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d6810c520 0x7f1d6810e9c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/862308653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 0x7f1d6810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/862308653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d681031e0 0x7f1d681035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.529+0000 7f1d6e3db640 1 -- 172.21.15.37:0/862308653 >> 172.21.15.37:0/862308653 conn(0x7f1d68077fd0 msgr2=0x7f1d680783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.530+0000 7f1d6e3db640 1 -- 172.21.15.37:0/862308653 shutdown_connections 2024-04-25T21:24:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.530+0000 7f1d6e3db640 1 -- 172.21.15.37:0/862308653 wait complete. 2024-04-25T21:24:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.530+0000 7f1d6e3db640 1 Processor -- start 2024-04-25T21:24:43.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.530+0000 7f1d6e3db640 1 -- start start 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 0x7f1d68101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6810c520 0x7f1d681018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68112190 con 0x7f1d681031e0 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68112010 con 0x7f1d6810c520 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6e3db640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68112310 con 0x7f1d68103b00 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45780/0 (socket says 172.21.15.37:45780) 2024-04-25T21:24:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 -- 172.21.15.37:0/1677576015 learned_addr learned my addr 172.21.15.37:0/1677576015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6810c520 0x7f1d681018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d677fe640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 0x7f1d68101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 -- 172.21.15.37:0/1677576015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 msgr2=0x7f1d68101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 0x7f1d68101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 -- 172.21.15.37:0/1677576015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6810c520 msgr2=0x7f1d681018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6810c520 0x7f1d681018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.533 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 -- 172.21.15.37:0/1677576015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d681a6a50 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d67fff640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c920 tx=0x7f1d5800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d657fa640 1 -- 172.21.15.37:0/1677576015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58007bb0 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.531+0000 7f1d657fa640 1 -- 172.21.15.37:0/1677576015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d58007d50 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.532+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d681a6d40 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.532+0000 7f1d657fa640 1 -- 172.21.15.37:0/1677576015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58018730 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.532+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d681a7230 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d657fa640 1 -- 172.21.15.37:0/1677576015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d580188d0 con 0x7f1d681031e0 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 Processor -- start 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 -- start start 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab7396a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ab72fb970 con 0x564ab7396f40 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564ab72984a0 con 0x564ab73a1800 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.534+0000 7f1d711cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564ab72f7930 con 0x564ab7396620 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45794/0 (socket says 172.21.15.37:45794) 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d67fff640 1 -- 172.21.15.37:0/1582479852 learned_addr learned my addr 172.21.15.37:0/1582479852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab7396a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d677fe640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d657fa640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 0x7f1d500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 -- 172.21.15.37:0/1582479852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 msgr2=0x564ab73a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 -- 172.21.15.37:0/1582479852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 msgr2=0x564ab73a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 -- 172.21.15.37:0/1582479852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab700e210 con 0x564ab7396620 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d67fff640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 0x7f1d500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d657fa640 1 -- 172.21.15.37:0/1677576015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f1d580d0930 con 0x7f1d681031e0 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.535+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab7396a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002af00 tx=0x7f1d6002ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ba9bfd5fc3ca5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d6c978640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 0x7f1d500a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6004c000 tx=0x7f1d60049000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d4e7fc640 1 -- 172.21.15.37:0/1582479852 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60033040 con 0x564ab7396620 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d4e7fc640 1 -- 172.21.15.37:0/1582479852 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6004a040 con 0x564ab7396620 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d4e7fc640 1 -- 172.21.15.37:0/1582479852 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60046610 con 0x564ab7396620 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 -- 172.21.15.37:0/1582479852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 msgr2=0x564ab7396a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab7396a00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002af00 tx=0x7f1d6002ecf0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 -- 172.21.15.37:0/1582479852 shutdown_connections 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396f40 0x564ab73a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 --2- 172.21.15.37:0/1582479852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab7396a00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.536+0000 7f1d711cd740 1 -- 172.21.15.37:0/1582479852 >> 172.21.15.37:0/1582479852 conn(0x564ab7392b30 msgr2=0x564ab7392f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- 172.21.15.37:0/1582479852 shutdown_connections 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- 172.21.15.37:0/1582479852 wait complete. 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- start start 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 Processor -- start 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab73a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab739e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564ab73a4430 con 0x564ab73a1800 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x564ab72960b0 con 0x564ab7396f40 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x564ab73a45b0 con 0x564ab7396620 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.537+0000 7f1d711cd740 10 client.0 _ll_register_callbacks cb 0x564ab7283d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab73a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45804/0 (socket says 172.21.15.37:45804) 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 -- 172.21.15.37:0/2466108464 learned_addr learned my addr 172.21.15.37:0/2466108464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d677fe640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab739e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 -- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 msgr2=0x564ab73a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab73a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 -- 172.21.15.37:0/2466108464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 msgr2=0x564ab739e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab739e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 -- 172.21.15.37:0/2466108464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab70da620 con 0x564ab73a1800 2024-04-25T21:24:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d67fff640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab73a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d6c978640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002ea40 tx=0x7f1d6002ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6946e7dbb76397d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d477fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6005e060 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f1d60038450 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.538+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d6004a040 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d60038690 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 10 client.17253 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ab73a4430 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d6004a040 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1d60050bf0 con 0x564ab73a1800 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 10 client.17253 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 20 client.17253 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab73b1540 0x564ab73b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 4 client.17253 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab7309390 con 0x564ab73b1540 2024-04-25T21:24:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.539+0000 7f1d67fff640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab73b1540 0x564ab73b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.540+0000 7f1d67fff640 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab73b1540 0x564ab73b3930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e9d2cc2e7587313 server_cookie=f52541b7ed69f4c8 in_seq=0 out_seq=0 2024-04-25T21:24:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.540+0000 7f1d477fe640 10 client.17253 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d477fe640 1 -- 172.21.15.37:0/2466108464 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ab7309390 con 0x564ab73b1540 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d477fe640 10 client.17253 handle_command_reply: tid=0 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 client.17253 shutdown 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 msgr2=0x564ab739e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1d6002ea40 tx=0x7f1d6002ff80 comp rx=0 tx=0).stop 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 shutdown_connections 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab73b1540 0x564ab73b3930 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab73a1800 0x564ab739e9f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab739e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2466108464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396620 0x564ab73a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 >> 172.21.15.37:0/2466108464 conn(0x564ab7392b30 msgr2=0x564ab73988b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d477fe640 0 client.17253 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 shutdown_connections 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 1 -- 172.21.15.37:0/2466108464 wait complete. 2024-04-25T21:24:43.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.541+0000 7f1d711cd740 20 client.17253 trim_cache size 0 max 16384 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 Processor -- start 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 -- start start 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681afc10 con 0x564ab7396620 2024-04-25T21:24:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681b1750 con 0x564ab73a1800 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.557+0000 7f1d711cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681b02e0 con 0x564ab7396f40 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45816/0 (socket says 172.21.15.37:45816) 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:57038/0 (socket says 172.21.15.37:57038) 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d67fff640 1 -- 172.21.15.37:0/2666750637 learned_addr learned my addr 172.21.15.37:0/2666750637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d6c978640 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 -- 172.21.15.37:0/2666750637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 msgr2=0x564ab73b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 -- 172.21.15.37:0/2666750637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 msgr2=0x564ab73ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 -- 172.21.15.37:0/2666750637 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab73923c0 con 0x564ab7396f40 2024-04-25T21:24:43.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d6c978640 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:43.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d677fe640 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x564ab739dcd0 tx=0x7f1d5c00a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7847dd7b8b56668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d4dffb640 1 -- 172.21.15.37:0/2666750637 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01e030 con 0x564ab7396f40 2024-04-25T21:24:43.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d4dffb640 1 -- 172.21.15.37:0/2666750637 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c0054d0 con 0x564ab7396f40 2024-04-25T21:24:43.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.558+0000 7f1d4dffb640 1 -- 172.21.15.37:0/2666750637 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01a400 con 0x564ab7396f40 2024-04-25T21:24:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- 172.21.15.37:0/2666750637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 msgr2=0x564ab73b90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x564ab739dcd0 tx=0x7f1d5c00a2d0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- 172.21.15.37:0/2666750637 shutdown_connections 2024-04-25T21:24:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab73a1800 0x564ab73b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab7396f40 0x564ab73b90c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 --2- 172.21.15.37:0/2666750637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- 172.21.15.37:0/2666750637 >> 172.21.15.37:0/2666750637 conn(0x564ab7392b30 msgr2=0x564ab7391cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- 172.21.15.37:0/2666750637 shutdown_connections 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- 172.21.15.37:0/2666750637 wait complete. 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.559+0000 7f1d711cd740 1 -- start start 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 Processor -- start 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab73a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 0x564ab73a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681b1080 con 0x564ab7396620 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681b09b0 con 0x564ab7396f40 2024-04-25T21:24:43.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f1d681adc10 con 0x564ab73a1800 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d711cd740 10 client.0 _ll_register_callbacks cb 0x7f1d6002ec20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45824/0 (socket says 172.21.15.37:45824) 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d67fff640 1 -- 172.21.15.37:0/37321404 learned_addr learned my addr 172.21.15.37:0/37321404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d6c978640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 0x564ab73a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d677fe640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab73a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d67fff640 1 -- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 msgr2=0x564ab73a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.560+0000 7f1d67fff640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 0x564ab73a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d67fff640 1 -- 172.21.15.37:0/37321404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 msgr2=0x564ab73a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d67fff640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab73a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d67fff640 1 -- 172.21.15.37:0/37321404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab7416a40 con 0x564ab7396620 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d677fe640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab73a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d6c978640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 0x564ab73a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:24:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d67fff640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1d58087010 tx=0x7f1d580bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cc8b450666f104 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5809c8b0 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f1d5800f990 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5c0054d0 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1d580a0c60 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 10 client.17259 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ab7417db0 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c0054d0 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1d580ce370 con 0x564ab7396620 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 10 client.17259 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 20 client.17259 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.561+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab7418510 0x564ab741a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:43.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.562+0000 7f1d711cd740 4 client.17259 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:24:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.562+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ab7303200 con 0x564ab7418510 2024-04-25T21:24:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.562+0000 7f1d677fe640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab7418510 0x564ab741a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.562+0000 7f1d477fe640 10 client.17259 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:43.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.562+0000 7f1d677fe640 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab7418510 0x564ab741a930 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc9c2c626fffa2a server_cookie=9c724339b17088c9 in_seq=0 out_seq=0 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.573+0000 7f1d477fe640 1 -- 172.21.15.37:0/37321404 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ab7303200 con 0x564ab7418510 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.573+0000 7f1d477fe640 10 client.17259 handle_command_reply: tid=0 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.573+0000 7f1d711cd740 1 client.17259 shutdown 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 msgr2=0x564ab73a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f1d58087010 tx=0x7f1d580bd040 comp rx=0 tx=0).stop 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 shutdown_connections 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x564ab7418510 0x564ab741a930 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564ab73a1800 0x564ab73a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d477fe640 0 client.17259 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x564ab7396f40 0x564ab73a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 --2- 172.21.15.37:0/37321404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564ab7396620 0x564ab73a7350 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 >> 172.21.15.37:0/37321404 conn(0x564ab7392b30 msgr2=0x564ab7391cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 shutdown_connections 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 1 -- 172.21.15.37:0/37321404 wait complete. 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d711cd740 20 client.17259 trim_cache size 0 max 16384 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:24:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:24:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:24:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 msgr2=0x7f1d500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 0x7f1d500a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6004c000 tx=0x7f1d60049000 comp rx=0 tx=0).stop 2024-04-25T21:24:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 msgr2=0x7f1d68100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.574+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c920 tx=0x7f1d5800cd20 comp rx=0 tx=0).stop 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 shutdown_connections 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1d6810c520 0x7f1d681018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f1d500a3dc0 0x7f1d500a6280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d68103b00 0x7f1d68101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 --2- 172.21.15.37:0/1677576015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d681031e0 0x7f1d68100e30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 >> 172.21.15.37:0/1677576015 conn(0x7f1d68077fd0 msgr2=0x7f1d680fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 shutdown_connections 2024-04-25T21:24:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:43.575+0000 7f1d6e3db640 1 -- 172.21.15.37:0/1677576015 wait complete. 2024-04-25T21:24:43.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:43 smithi037 ceph-mon[29195]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:24:44.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:43 smithi177 ceph-mon[33043]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:24:44.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:43 smithi098 ceph-mon[32569]: pgmap v510: 97 pgs: 97 active+clean; 3.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.2 KiB/s wr, 11 op/s 2024-04-25T21:24:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 -- 172.21.15.37:0/1354116399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102300 msgr2=0x7fd87810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 --2- 172.21.15.37:0/1354116399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102300 0x7fd87810e7f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd870009990 tx=0x7fd870009ca0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 -- 172.21.15.37:0/1354116399 shutdown_connections 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 --2- 172.21.15.37:0/1354116399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878102300 0x7fd87810e7f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 --2- 172.21.15.37:0/1354116399 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878107990 0x7fd878107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 --2- 172.21.15.37:0/1354116399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878101940 0x7fd878101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 -- 172.21.15.37:0/1354116399 >> 172.21.15.37:0/1354116399 conn(0x7fd8780788f0 msgr2=0x7fd878078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 -- 172.21.15.37:0/1354116399 shutdown_connections 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.329+0000 7fd87f11c700 1 -- 172.21.15.37:0/1354116399 wait complete. 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 Processor -- start 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 -- start start 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 0x7fd87819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 0x7fd8781a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd878111b50 con 0x7fd878101940 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781119d0 con 0x7fd878102300 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87f11c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd878111cd0 con 0x7fd878107990 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87ceb8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87ceb8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45848/0 (socket says 172.21.15.37:45848) 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.330+0000 7fd87ceb8700 1 -- 172.21.15.37:0/858821560 learned_addr learned my addr 172.21.15.37:0/858821560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87d6b9700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 0x7fd8781a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd877fff700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 0x7fd87819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 -- 172.21.15.37:0/858821560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 msgr2=0x7fd8781a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 0x7fd8781a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 -- 172.21.15.37:0/858821560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 msgr2=0x7fd87819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 0x7fd87819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 -- 172.21.15.37:0/858821560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8781a5530 con 0x7fd878101940 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87d6b9700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 0x7fd8781a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd877fff700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 0x7fd87819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87ceb8700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c920 tx=0x7fd86000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860007b00 con 0x7fd878101940 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8781a5700 con 0x7fd878101940 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd860004ce0 con 0x7fd878101940 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860005650 con 0x7fd878101940 2024-04-25T21:24:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.331+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781a5c40 con 0x7fd878101940 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.332+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd878069730 con 0x7fd878101940 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.334+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd860004750 con 0x7fd878101940 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.334+0000 7fd875ffb700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 0x7fd8640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.335+0000 7fd877fff700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 0x7fd8640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.335+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fd8600d0830 con 0x7fd878101940 2024-04-25T21:24:45.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.335+0000 7fd877fff700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 0x7fd8640a6270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd878078d10 tx=0x7fd8680073c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.338+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd860014030 con 0x7fd878101940 2024-04-25T21:24:45.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.593+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd87806ada0 con 0x7fd878101940 2024-04-25T21:24:45.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.594+0000 7fd875ffb700 1 -- 172.21.15.37:0/858821560 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fd87806ada0 con 0x7fd878101940 2024-04-25T21:24:45.594 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:24:45.594 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 msgr2=0x7fd8640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 0x7fd8640a6270 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd878078d10 tx=0x7fd8680073c0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 msgr2=0x7fd878199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c920 tx=0x7fd86000cce0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 shutdown_connections 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd8640a3db0 0x7fd8640a6270 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd878107990 0x7fd8781a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd878101940 0x7fd878199f70 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 --2- 172.21.15.37:0/858821560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd878102300 0x7fd87819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 >> 172.21.15.37:0/858821560 conn(0x7fd8780788f0 msgr2=0x7fd8780ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 shutdown_connections 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:45.596+0000 7fd87f11c700 1 -- 172.21.15.37:0/858821560 wait complete. 2024-04-25T21:24:45.597 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:24:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:45 smithi037 ceph-mon[29195]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:24:45.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/858821560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:45 smithi177 ceph-mon[33043]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:24:46.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/858821560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:46.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:45 smithi098 ceph-mon[32569]: pgmap v511: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-25T21:24:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/858821560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:24:46.389 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --set-id 45c365d2 --release --await' 2024-04-25T21:24:46.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --set-id 45c365d2 --release --await 2024-04-25T21:24:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:24:48.012 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:47 smithi037 ceph-mon[29195]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:48.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:47 smithi177 ceph-mon[33043]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:48.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:47 smithi098 ceph-mon[32569]: pgmap v512: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 -- 172.21.15.37:0/4120636836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 msgr2=0x7f085410f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 --2- 172.21.15.37:0/4120636836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f085410f830 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b0a0 tx=0x7f084c00b3b0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 -- 172.21.15.37:0/4120636836 shutdown_connections 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 --2- 172.21.15.37:0/4120636836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f085410f830 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 --2- 172.21.15.37:0/4120636836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f0854107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 --2- 172.21.15.37:0/4120636836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 0x7f0854101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 -- 172.21.15.37:0/4120636836 >> 172.21.15.37:0/4120636836 conn(0x7f0854076c60 msgr2=0x7f0854077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 -- 172.21.15.37:0/4120636836 shutdown_connections 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.493+0000 7f0858cd9700 1 -- 172.21.15.37:0/4120636836 wait complete. 2024-04-25T21:24:48.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 Processor -- start 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 -- start start 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 0x7f08541990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f085419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0854112c70 con 0x7f0854101dc0 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0854112af0 con 0x7f0854101400 2024-04-25T21:24:48.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0858cd9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0854112df0 con 0x7f0854107b20 2024-04-25T21:24:48.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45856/0 (socket says 172.21.15.37:45856) 2024-04-25T21:24:48.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f0851d9b700 1 -- 172.21.15.37:0/197742451 learned_addr learned my addr 172.21.15.37:0/197742451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.494+0000 7f085259c700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 0x7f08541990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0852d9d700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f085419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 -- 172.21.15.37:0/197742451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 msgr2=0x7f085419d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f085419d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 -- 172.21.15.37:0/197742451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 msgr2=0x7f08541990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 0x7f08541990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 -- 172.21.15.37:0/197742451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085419e060 con 0x7f0854101dc0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0852d9d700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f085419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0851d9b700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f084800d980 tx=0x7f084800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f08437fe700 1 -- 172.21.15.37:0/197742451 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848016070 con 0x7f0854101dc0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085419e350 con 0x7f0854101dc0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f08437fe700 1 -- 172.21.15.37:0/197742451 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848004830 con 0x7f0854101dc0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f08437fe700 1 -- 172.21.15.37:0/197742451 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480054c0 con 0x7f0854101dc0 2024-04-25T21:24:48.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.495+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08541a5bd0 con 0x7f0854101dc0 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.497+0000 7f08437fe700 1 -- 172.21.15.37:0/197742451 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0848004220 con 0x7f0854101dc0 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.498+0000 7f08437fe700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 0x7f083c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.498+0000 7f085259c700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 0x7f083c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.498+0000 7f08437fe700 1 -- 172.21.15.37:0/197742451 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f08480d0670 con 0x7f0854101dc0 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.498+0000 7f085259c700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 0x7f083c0a6270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08540781e0 tx=0x7f084400b410 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 Processor -- start 2024-04-25T21:24:48.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 -- start start 2024-04-25T21:24:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055650 0x5572a9055a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 0x5572a9067670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5572a8fa8d90 con 0x5572a9055f70 2024-04-25T21:24:48.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5572a8f4fa30 con 0x5572a9065280 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.499+0000 7f0863562b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5572a8fa4d50 con 0x5572a9055650 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45870/0 (socket says 172.21.15.37:45870) 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 -- 172.21.15.37:0/2111274952 learned_addr learned my addr 172.21.15.37:0/2111274952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f085259c700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055650 0x5572a9055a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0852d9d700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 0x5572a9067670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 -- 172.21.15.37:0/2111274952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055650 msgr2=0x5572a9055a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055650 0x5572a9055a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 -- 172.21.15.37:0/2111274952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 msgr2=0x5572a9067670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 0x5572a9067670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 -- 172.21.15.37:0/2111274952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a8f13840 con 0x5572a9055f70 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0852d9d700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 0x5572a9067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f08480a4d50 tx=0x7f08480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd1c811ec0870dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0840ff9700 1 -- 172.21.15.37:0/2111274952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08480a09c0 con 0x5572a9055f70 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0840ff9700 1 -- 172.21.15.37:0/2111274952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848098040 con 0x5572a9055f70 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.500+0000 7f0840ff9700 1 -- 172.21.15.37:0/2111274952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084809c870 con 0x5572a9055f70 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 -- 172.21.15.37:0/2111274952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 msgr2=0x5572a9064d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f08480a4d50 tx=0x7f08480045e0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 -- 172.21.15.37:0/2111274952 shutdown_connections 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9065280 0x5572a9067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055650 0x5572a9055a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 --2- 172.21.15.37:0/2111274952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9064d40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 -- 172.21.15.37:0/2111274952 >> 172.21.15.37:0/2111274952 conn(0x5572a8f82990 msgr2=0x5572a9051b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 -- 172.21.15.37:0/2111274952 shutdown_connections 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.501+0000 7f0863562b80 1 -- 172.21.15.37:0/2111274952 wait complete. 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 -- start start 2024-04-25T21:24:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 Processor -- start 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 0x5572a9061db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a90622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5572a9067cb0 con 0x5572a9055650 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5572a8f626b0 con 0x5572a9055f70 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x5572a9067e30 con 0x5572a9065280 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0863562b80 10 client.0 _ll_register_callbacks cb 0x5572a8f41a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45886/0 (socket says 172.21.15.37:45886) 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 -- 172.21.15.37:0/174906851 learned_addr learned my addr 172.21.15.37:0/174906851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0851d9b700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 0x5572a9061db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0852d9d700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a90622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 -- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 msgr2=0x5572a90622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a90622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 -- 172.21.15.37:0/174906851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 msgr2=0x5572a9061db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 0x5572a9061db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f085259c700 1 -- 172.21.15.37:0/174906851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a8ef69d0 con 0x5572a9055650 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0852d9d700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a90622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.502+0000 7f0851d9b700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 0x5572a9061db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f085259c700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f08440116c0 tx=0x7f08440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa585ed3161534eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844028670 con 0x5572a9055650 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f0844028810 con 0x5572a9055650 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0848098040 con 0x5572a9055650 2024-04-25T21:24:48.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0844028a10 con 0x5572a9055650 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 10 client.17277 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572a9072600 con 0x5572a9055650 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0848098040 con 0x5572a9055650 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0844031b50 con 0x5572a9055650 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 10 client.17277 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 10 client.17277 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 20 client.17277 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a9074b90 0x5572a9076fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 4 client.17277 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.503+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572a8fb6580 con 0x5572a9074b90 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.504+0000 7f0851d9b700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a9074b90 0x5572a9076fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.504+0000 7f0851d9b700 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a9074b90 0x5572a9076fc0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23770e0e2cc8809 server_cookie=3935022da970e947 in_seq=0 out_seq=0 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.504+0000 7f0839ffb700 10 client.17277 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:48.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0839ffb700 1 -- 172.21.15.37:0/174906851 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572a8fb6580 con 0x5572a9074b90 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0839ffb700 10 client.17277 handle_command_reply: tid=0 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 client.17277 shutdown 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 msgr2=0x5572a9064ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f08440116c0 tx=0x7f08440047c0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 shutdown_connections 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a9074b90 0x5572a9076fc0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a90622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055650 0x5572a9064ab0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0839ffb700 0 client.17277 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 --2- 172.21.15.37:0/174906851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055f70 0x5572a9061db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 >> 172.21.15.37:0/174906851 conn(0x5572a8f82990 msgr2=0x5572a9057ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 shutdown_connections 2024-04-25T21:24:48.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 1 -- 172.21.15.37:0/174906851 wait complete. 2024-04-25T21:24:48.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.505+0000 7f0863562b80 20 client.17277 trim_cache size 0 max 16384 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 Processor -- start 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 -- start start 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a9057fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055f70 0x5572a90645f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ab1b0 con 0x5572a9065280 2024-04-25T21:24:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08541accf0 con 0x5572a9055650 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.525+0000 7f0863562b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ab880 con 0x5572a9055f70 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45898/0 (socket says 172.21.15.37:45898) 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 -- 172.21.15.37:0/3186993585 learned_addr learned my addr 172.21.15.37:0/3186993585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0851d9b700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055f70 0x5572a90645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 -- 172.21.15.37:0/3186993585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055f70 msgr2=0x5572a90645f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055f70 0x5572a90645f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f085259c700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a9057fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 -- 172.21.15.37:0/3186993585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 msgr2=0x5572a9057fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a9057fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 -- 172.21.15.37:0/3186993585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572a9077640 con 0x5572a9065280 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f085259c700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a9057fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f0852d9d700 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b830 tx=0x7f084c009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b45ddcf4645ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f083bfff700 1 -- 172.21.15.37:0/3186993585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c009450 con 0x5572a9065280 2024-04-25T21:24:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f083bfff700 1 -- 172.21.15.37:0/3186993585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f084c0074e0 con 0x5572a9065280 2024-04-25T21:24:48.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.526+0000 7f083bfff700 1 -- 172.21.15.37:0/3186993585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0052a0 con 0x5572a9065280 2024-04-25T21:24:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- 172.21.15.37:0/3186993585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 msgr2=0x5572a907c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b830 tx=0x7f084c009d10 comp rx=0 tx=0).stop 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- 172.21.15.37:0/3186993585 shutdown_connections 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9065280 0x5572a907c890 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a9057fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 --2- 172.21.15.37:0/3186993585 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9055f70 0x5572a90645f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- 172.21.15.37:0/3186993585 >> 172.21.15.37:0/3186993585 conn(0x5572a8f82990 msgr2=0x5572a9050d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- 172.21.15.37:0/3186993585 shutdown_connections 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- 172.21.15.37:0/3186993585 wait complete. 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 -- start start 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.527+0000 7f0863562b80 1 Processor -- start 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a906d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a9069c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ac620 con 0x5572a9055f70 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08541abf50 con 0x5572a9055650 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f08541a9170 con 0x5572a9065280 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 10 client.0 _ll_register_callbacks cb 0x5572a8f41a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45914/0 (socket says 172.21.15.37:45914) 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 -- 172.21.15.37:0/2348911354 learned_addr learned my addr 172.21.15.37:0/2348911354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f085259c700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a906d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0852d9d700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a9069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 -- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 msgr2=0x5572a9069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a9069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 -- 172.21.15.37:0/2348911354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 msgr2=0x5572a906d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a906d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 -- 172.21.15.37:0/2348911354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572a906a360 con 0x5572a9055f70 2024-04-25T21:24:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0851d9b700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f08480cde70 tx=0x7f0848007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d6e0ca4d27b7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0839ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848066060 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7f0848005660 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f084c0074e0 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.528+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08480988b0 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 10 client.17283 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572a90dc000 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f084c0074e0 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f08480a4df0 con 0x5572a9055f70 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 10 client.17283 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 10 client.17283 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 20 client.17283 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a90dc2f0 0x5572a90de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 4 client.17283 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "45c365d2", "release": true, "await": true}] 2024-04-25T21:24:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "set_id": "45c365d2", "release": true, "await": true}) -- 0x5572a8fb0620 con 0x5572a90dc2f0 2024-04-25T21:24:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.529+0000 7f085259c700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a90dc2f0 0x5572a90de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.530+0000 7f0839ffb700 10 client.17283 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.530+0000 7f085259c700 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a90dc2f0 0x5572a90de6e0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8edf0a5171364f server_cookie=f639467c7ba3e430 in_seq=0 out_seq=0 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.533+0000 7f0839ffb700 1 -- 172.21.15.37:0/2348911354 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5572a8fb0620 con 0x5572a90dc2f0 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.533+0000 7f0839ffb700 10 client.17283 handle_command_reply: tid=0 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 client.17283 shutdown 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 msgr2=0x5572a9069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f08480cde70 tx=0x7f0848007640 comp rx=0 tx=0).stop 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 shutdown_connections 2024-04-25T21:24:48.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x5572a90dc2f0 0x5572a90de6e0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5572a9065280 0x5572a9069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x5572a9055650 0x5572a906d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 --2- 172.21.15.37:0/2348911354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572a9055f70 0x5572a9069710 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 >> 172.21.15.37:0/2348911354 conn(0x5572a8f82990 msgr2=0x5572a9057fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0839ffb700 0 client.17283 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 shutdown_connections 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 1 -- 172.21.15.37:0/2348911354 wait complete. 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.534+0000 7f0863562b80 20 client.17283 trim_cache size 0 max 16384 2024-04-25T21:24:48.536 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 44, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "45c365d2": { 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 44, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:24:48.537 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 84.2, 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:48.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:24:48.539 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:24:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 msgr2=0x7f083c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 0x7f083c0a6270 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08540781e0 tx=0x7f084400b410 comp rx=0 tx=0).stop 2024-04-25T21:24:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 msgr2=0x7f08541995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f084800d980 tx=0x7f084800dc90 comp rx=0 tx=0).stop 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 shutdown_connections 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f083c0a3db0 0x7f083c0a6270 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0854107b20 0x7f085419d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0854101400 0x7f08541990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 --2- 172.21.15.37:0/197742451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0854101dc0 0x7f08541995e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 >> 172.21.15.37:0/197742451 conn(0x7f0854076c60 msgr2=0x7f08540fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 shutdown_connections 2024-04-25T21:24:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:24:48.535+0000 7f0858cd9700 1 -- 172.21.15.37:0/197742451 wait complete. 2024-04-25T21:24:49.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '45c365d2', seconds elapsed: 36.5 2024-04-25T21:24:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:49 smithi177 ceph-mon[33043]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:50.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:49 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:24:50.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:49 smithi098 ceph-mon[32569]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:50.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:49 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:24:50.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:49 smithi037 ceph-mon[29195]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2024-04-25T21:24:50.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:49 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:24:51.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:50 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:24:51.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:50 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:24:51.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:50 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:24:51.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:50 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:24:51.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:50 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:24:51.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:50 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:24:51.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:50 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:24:51.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:50 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:24:51.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:50 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:24:52.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:51 smithi177 ceph-mon[33043]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T21:24:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:51 smithi098 ceph-mon[32569]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T21:24:52.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:51 smithi037 ceph-mon[29195]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 KiB/s wr, 15 op/s 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:499/600: open fd = 3 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:500/600: open fd = 3 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:501/600: open fd = 3 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.571 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:502/600: open fd = 3 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:503/600: open fd = 3 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:504/600: open fd = 3 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:505/600: open fd = 3 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.572 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:506/600: open fd = 3 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:507/600: open fd = 3 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:508/600: open fd = 3 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.573 INFO:tasks.workunit.client.0.smithi037.stdout:509/600: open fd = 3 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:510/600: open fd = 3 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:511/600: open fd = 3 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:512/600: open fd = 3 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.574 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:513/600: open fd = 3 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:514/600: open fd = 3 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:515/600: open fd = 3 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:516/600: open fd = 3 2024-04-25T21:24:52.575 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:517/600: open fd = 3 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:518/600: open fd = 3 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:519/600: open fd = 3 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.576 INFO:tasks.workunit.client.0.smithi037.stdout:520/600: open fd = 3 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:521/600: open fd = 3 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:522/600: open fd = 3 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:523/600: open fd = 3 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.577 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:524/600: open fd = 3 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:525/600: open fd = 3 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:526/600: open fd = 3 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:527/600: open fd = 3 2024-04-25T21:24:52.578 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:528/600: open fd = 3 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:529/600: open fd = 3 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:530/600: open fd = 3 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.579 INFO:tasks.workunit.client.0.smithi037.stdout:531/600: open fd = 3 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:532/600: open fd = 3 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:533/600: open fd = 3 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:534/600: open fd = 3 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.580 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:535/600: open fd = 3 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:536/600: open fd = 3 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:537/600: open fd = 3 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:538/600: open fd = 3 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.581 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:539/600: open fd = 3 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:540/600: open fd = 3 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:541/600: open fd = 3 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:542/600: open fd = 3 2024-04-25T21:24:52.582 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:543/600: open fd = 3 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:544/600: open fd = 3 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:545/600: open fd = 3 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.583 INFO:tasks.workunit.client.0.smithi037.stdout:546/600: open fd = 3 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:547/600: open fd = 3 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:548/600: open fd = 3 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:549/600: open fd = 3 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.584 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:550/600: open fd = 3 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:551/600: open fd = 3 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:552/600: open fd = 3 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:553/600: open fd = 3 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.585 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:554/600: open fd = 3 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:555/600: open fd = 3 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:556/600: open fd = 3 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.586 INFO:tasks.workunit.client.0.smithi037.stdout:557/600: open fd = 3 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:558/600: open fd = 3 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:559/600: open fd = 3 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:560/600: open fd = 3 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:24:52.587 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:24:54.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:53 smithi177 ceph-mon[33043]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T21:24:54.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:53 smithi098 ceph-mon[32569]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T21:24:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:53 smithi037 ceph-mon[29195]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-25T21:24:56.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:55 smithi177 ceph-mon[33043]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T21:24:56.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:55 smithi098 ceph-mon[32569]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T21:24:56.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:55 smithi037 ceph-mon[29195]: pgmap v516: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/1524974283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 msgr2=0x7fdb741014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/1524974283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb741014d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdb64009900 tx=0x7fdb6402a3f0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/1524974283 shutdown_connections 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/1524974283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb74106170 0x7fdb7410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/1524974283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74101aa0 0x7fdb74105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/1524974283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb741014d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/1524974283 >> 172.21.15.37:0/1524974283 conn(0x7fdb74077fd0 msgr2=0x7fdb740783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/1524974283 shutdown_connections 2024-04-25T21:24:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/1524974283 wait complete. 2024-04-25T21:24:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 Processor -- start 2024-04-25T21:24:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.676+0000 7fdb7b1d6640 1 -- start start 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74106170 0x7fdb74198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb7419cf20 0x7fdb7419d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741110d0 con 0x7fdb741010f0 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74110f50 con 0x7fdb7419cf20 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb7b1d6640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111250 con 0x7fdb74106170 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59754/0 (socket says 172.21.15.37:59754) 2024-04-25T21:24:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 -- 172.21.15.37:0/2482743489 learned_addr learned my addr 172.21.15.37:0/2482743489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb79773640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb7419cf20 0x7fdb7419d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb73fff640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74106170 0x7fdb74198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 -- 172.21.15.37:0/2482743489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74106170 msgr2=0x7fdb74198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74106170 0x7fdb74198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 -- 172.21.15.37:0/2482743489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb7419cf20 msgr2=0x7fdb7419d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb7419cf20 0x7fdb7419d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 -- 172.21.15.37:0/2482743489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7419da10 con 0x7fdb741010f0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.677+0000 7fdb78f72640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdb64005940 tx=0x7fdb640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.678+0000 7fdb71ffb640 1 -- 172.21.15.37:0/2482743489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6403f030 con 0x7fdb741010f0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.678+0000 7fdb71ffb640 1 -- 172.21.15.37:0/2482743489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb64003cb0 con 0x7fdb741010f0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.678+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7419dca0 con 0x7fdb741010f0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.678+0000 7fdb71ffb640 1 -- 172.21.15.37:0/2482743489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64035070 con 0x7fdb741010f0 2024-04-25T21:24:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.678+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a5460 con 0x7fdb741010f0 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.680+0000 7fdb71ffb640 1 -- 172.21.15.37:0/2482743489 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb64042030 con 0x7fdb741010f0 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 Processor -- start 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 -- start start 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0df40 0x556e25e182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e18800 0x556e25e1abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e25d72970 con 0x556e25e0d620 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e25d0f4a0 con 0x556e25e0df40 2024-04-25T21:24:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e25d6e930 con 0x556e25e18800 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb71ffb640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 0x7fdb5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59760/0 (socket says 172.21.15.37:59760) 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 -- 172.21.15.37:0/2619591883 learned_addr learned my addr 172.21.15.37:0/2619591883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 0x7fdb5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb71ffb640 1 -- 172.21.15.37:0/2482743489 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fdb640f0ff0 con 0x7fdb741010f0 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb73fff640 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e18800 0x556e25e1abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb78f72640 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0df40 0x556e25e182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 -- 172.21.15.37:0/2619591883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e18800 msgr2=0x556e25e1abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e18800 0x556e25e1abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 -- 172.21.15.37:0/2619591883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0df40 msgr2=0x556e25e182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0df40 0x556e25e182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.681+0000 7fdb79773640 1 -- 172.21.15.37:0/2619591883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e25a85210 con 0x556e25e0d620 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.682+0000 7fdb79773640 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c014e80 tx=0x7fdb6c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7681828e87753e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.682+0000 7fdb79773640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 0x7fdb5c0a6280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c011cb0 tx=0x7fdb6c016000 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.682+0000 7fdb5affd640 1 -- 172.21.15.37:0/2619591883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c024070 con 0x556e25e0d620 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.682+0000 7fdb5affd640 1 -- 172.21.15.37:0/2619591883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c007c60 con 0x556e25e0d620 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.682+0000 7fdb5affd640 1 -- 172.21.15.37:0/2619591883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c005770 con 0x556e25e0d620 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2619591883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 msgr2=0x556e25e0da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c014e80 tx=0x7fdb6c0117b0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2619591883 shutdown_connections 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e18800 0x556e25e1abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0df40 0x556e25e182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2619591883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e0da00 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2619591883 >> 172.21.15.37:0/2619591883 conn(0x556e25e09b30 msgr2=0x556e25e09f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2619591883 shutdown_connections 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2619591883 wait complete. 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 -- start start 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.683+0000 7fdb7dfc5740 1 Processor -- start 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e18800 0x556e25e15ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556e25e1b3d0 con 0x556e25e0d620 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x556e25d0d0b0 con 0x556e25e18800 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x556e25e1b550 con 0x556e25e0df40 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb7dfc5740 10 client.0 _ll_register_callbacks cb 0x556e25cfad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59774/0 (socket says 172.21.15.37:59774) 2024-04-25T21:24:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 -- 172.21.15.37:0/467411775 learned_addr learned my addr 172.21.15.37:0/467411775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb73fff640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb79773640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e18800 0x556e25e15ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 -- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 msgr2=0x556e25e154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 -- 172.21.15.37:0/467411775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e18800 msgr2=0x556e25e15ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e18800 0x556e25e15ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 -- 172.21.15.37:0/467411775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e25b51620 con 0x556e25e0d620 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb78f72640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdb640b9480 tx=0x7fdb640bc980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab48c012bd53b9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb53fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640d5070 con 0x556e25e0d620 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.684+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fdb640ee9a0 con 0x556e25e0d620 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6c007c60 con 0x556e25e0d620 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb64049050 con 0x556e25e0d620 2024-04-25T21:24:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 10 client.17295 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e25e1b3d0 con 0x556e25e0d620 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c007c60 con 0x556e25e0d620 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb6403bbf0 con 0x556e25e0d620 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 10 client.17295 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 10 client.17295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 20 client.17295 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e287f0 0x556e25e2ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 4 client.17295 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e25d80390 con 0x556e25e287f0 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.685+0000 7fdb73fff640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e287f0 0x556e25e2ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.686+0000 7fdb73fff640 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e287f0 0x556e25e2ac20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a0274f286c2717 server_cookie=448ccc64dd4bcf76 in_seq=0 out_seq=0 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.686+0000 7fdb53fff640 10 client.17295 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb53fff640 1 -- 172.21.15.37:0/467411775 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e25d80390 con 0x556e25e287f0 2024-04-25T21:24:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb53fff640 10 client.17295 handle_command_reply: tid=0 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 client.17295 shutdown 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 msgr2=0x556e25e181e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdb640b9480 tx=0x7fdb640bc980 comp rx=0 tx=0).stop 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 shutdown_connections 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e287f0 0x556e25e2ac20 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e18800 0x556e25e15ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb53fff640 0 client.17295 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/467411775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0d620 0x556e25e181e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 >> 172.21.15.37:0/467411775 conn(0x556e25e09b30 msgr2=0x556e25e0f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 shutdown_connections 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/467411775 wait complete. 2024-04-25T21:24:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.687+0000 7fdb7dfc5740 20 client.17295 trim_cache size 0 max 16384 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 Processor -- start 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 -- start start 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e09b30 0x556e25e24290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ad9e0 con 0x556e25e0df40 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741af520 con 0x556e25e0d620 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.702+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ae0b0 con 0x556e25e09b30 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59776/0 (socket says 172.21.15.37:59776) 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 -- 172.21.15.37:0/2522636891 learned_addr learned my addr 172.21.15.37:0/2522636891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb78f72640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e09b30 0x556e25e24290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb73fff640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 -- 172.21.15.37:0/2522636891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e09b30 msgr2=0x556e25e24290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e09b30 0x556e25e24290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 -- 172.21.15.37:0/2522636891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 msgr2=0x556e25e10c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e10c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 -- 172.21.15.37:0/2522636891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e25e15e10 con 0x556e25e0df40 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb73fff640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb79773640 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c014a30 tx=0x7fdb6c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a1d7fbaa5167f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb5a7fc640 1 -- 172.21.15.37:0/2522636891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c024040 con 0x556e25e0df40 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb5a7fc640 1 -- 172.21.15.37:0/2522636891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c035070 con 0x556e25e0df40 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.703+0000 7fdb5a7fc640 1 -- 172.21.15.37:0/2522636891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c005440 con 0x556e25e0df40 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2522636891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 msgr2=0x556e25e30500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c014a30 tx=0x7fdb6c005040 comp rx=0 tx=0).stop 2024-04-25T21:24:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2522636891 shutdown_connections 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e0df40 0x556e25e30500 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e10c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/2522636891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e09b30 0x556e25e24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2522636891 >> 172.21.15.37:0/2522636891 conn(0x556e25e0fd40 msgr2=0x556e25e08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2522636891 shutdown_connections 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/2522636891 wait complete. 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 -- start start 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.704+0000 7fdb7dfc5740 1 Processor -- start 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e126e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e12c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741aee50 con 0x556e25e09b30 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ae780 con 0x556e25e0d620 2024-04-25T21:24:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdb741ab9e0 con 0x556e25e0df40 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb7dfc5740 10 client.0 _ll_register_callbacks cb 0x7fdb640bcec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59782/0 (socket says 172.21.15.37:59782) 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 -- 172.21.15.37:0/1011576656 learned_addr learned my addr 172.21.15.37:0/1011576656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb73fff640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb79773640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e12c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 -- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 msgr2=0x556e25e12c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e12c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 -- 172.21.15.37:0/1011576656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 msgr2=0x556e25e126e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e126e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 -- 172.21.15.37:0/1011576656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e25e8de20 con 0x556e25e09b30 2024-04-25T21:24:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb73fff640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb79773640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e12c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb78f72640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdb64031440 tx=0x7fdb640bcab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607150c2dd168ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb53fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640d5040 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.705+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8513+0+0 (secure 0 0 0) 0x7fdb640ee9a0 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6c035070 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdb64005d50 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 10 client.17301 fetch_fsmap learned FSMap version 27 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e25e8f080 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb6c035070 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdb640c5750 con 0x556e25e09b30 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 10 client.17301 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:24:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 20 client.17301 populate_metadata read hostname 'smithi037' 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e18800 0x556e25e915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 4 client.17301 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e25d7a200 con 0x556e25e18800 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.706+0000 7fdb73fff640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e18800 0x556e25e915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.707+0000 7fdb53fff640 10 client.17301 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.707+0000 7fdb73fff640 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e18800 0x556e25e915d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5105837d3e1412b server_cookie=1867650ff69ac14f in_seq=0 out_seq=0 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.720+0000 7fdb53fff640 1 -- 172.21.15.37:0/1011576656 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e25d7a200 con 0x556e25e18800 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.720+0000 7fdb53fff640 10 client.17301 handle_command_reply: tid=0 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.720+0000 7fdb7dfc5740 1 client.17301 shutdown 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 msgr2=0x556e25e121a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdb64031440 tx=0x7fdb640bcab0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 shutdown_connections 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x556e25e18800 0x556e25e915d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x556e25e0df40 0x556e25e12c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x556e25e0d620 0x556e25e126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 --2- 172.21.15.37:0/1011576656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556e25e09b30 0x556e25e121a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb53fff640 0 client.17301 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 >> 172.21.15.37:0/1011576656 conn(0x556e25e0fd40 msgr2=0x556e25e23280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 shutdown_connections 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 1 -- 172.21.15.37:0/1011576656 wait complete. 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7dfc5740 20 client.17301 trim_cache size 0 max 16384 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:24:56.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 msgr2=0x7fdb5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 0x7fdb5c0a6280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c011cb0 tx=0x7fdb6c016000 comp rx=0 tx=0).stop 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 msgr2=0x7fdb74198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.721+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdb64005940 tx=0x7fdb640042f0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 shutdown_connections 2024-04-25T21:24:56.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb7419cf20 0x7fdb7419d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdb5c0a3dc0 0x7fdb5c0a6280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb74106170 0x7fdb74198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 --2- 172.21.15.37:0/2482743489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb741010f0 0x7fdb74198660 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 >> 172.21.15.37:0/2482743489 conn(0x7fdb74077fd0 msgr2=0x7fdb740698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 shutdown_connections 2024-04-25T21:24:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:24:56.722+0000 7fdb7b1d6640 1 -- 172.21.15.37:0/2482743489 wait complete. 2024-04-25T21:24:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:57 smithi177 ceph-mon[33043]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:24:58.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:57 smithi098 ceph-mon[32569]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:24:58.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:57 smithi037 ceph-mon[29195]: pgmap v517: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:25:00.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:24:59 smithi177 ceph-mon[33043]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:25:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:24:59 smithi098 ceph-mon[32569]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:25:00.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:24:59 smithi037 ceph-mon[29195]: pgmap v518: 97 pgs: 97 active+clean; 3.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2024-04-25T21:25:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:01 smithi177 ceph-mon[33043]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T21:25:02.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:01 smithi098 ceph-mon[32569]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T21:25:02.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:01 smithi037 ceph-mon[29195]: pgmap v519: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-25T21:25:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:02 smithi177 ceph-mon[33043]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T21:25:03.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:02 smithi098 ceph-mon[32569]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T21:25:03.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:02 smithi037 ceph-mon[29195]: osdmap e112: 12 total, 12 up, 12 in 2024-04-25T21:25:04.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:03 smithi177 ceph-mon[33043]: pgmap v521: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:25:04.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:03 smithi098 ceph-mon[32569]: pgmap v521: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:25:04.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:03 smithi037 ceph-mon[29195]: pgmap v521: 97 pgs: 97 active+clean; 3.6 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 18 op/s 2024-04-25T21:25:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:05 smithi177 ceph-mon[33043]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-25T21:25:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:05 smithi177 ceph-mon[33043]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T21:25:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:05 smithi098 ceph-mon[32569]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-25T21:25:06.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:05 smithi098 ceph-mon[32569]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T21:25:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:05 smithi037 ceph-mon[29195]: pgmap v522: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-25T21:25:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:05 smithi037 ceph-mon[29195]: osdmap e113: 12 total, 12 up, 12 in 2024-04-25T21:25:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:07 smithi177 ceph-mon[33043]: pgmap v524: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T21:25:08.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:07 smithi098 ceph-mon[32569]: pgmap v524: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T21:25:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:07 smithi037 ceph-mon[29195]: pgmap v524: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 128 KiB/s wr, 48 op/s 2024-04-25T21:25:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.822+0000 7f18a4320640 1 -- 172.21.15.37:0/2110423496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103b20 msgr2=0x7f189c10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.822+0000 7f18a4320640 1 --2- 172.21.15.37:0/2110423496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103b20 0x7f189c10c000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f188c009900 tx=0x7f188c02a480 comp rx=0 tx=0).stop 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 -- 172.21.15.37:0/2110423496 shutdown_connections 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 --2- 172.21.15.37:0/2110423496 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c10c5d0 0x7f189c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 --2- 172.21.15.37:0/2110423496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c103b20 0x7f189c10c000 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 --2- 172.21.15.37:0/2110423496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 0x7f189c1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 -- 172.21.15.37:0/2110423496 >> 172.21.15.37:0/2110423496 conn(0x7f189c0771d0 msgr2=0x7f189c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 -- 172.21.15.37:0/2110423496 shutdown_connections 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 -- 172.21.15.37:0/2110423496 wait complete. 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 Processor -- start 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 -- start start 2024-04-25T21:25:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.823+0000 7f18a4320640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 0x7f189c100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c103b20 0x7f189c101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1121f0 con 0x7f189c10c5d0 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f189c112070 con 0x7f189c103b20 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f189c112370 con 0x7f189c103200 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52214/0 (socket says 172.21.15.37:52214) 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 -- 172.21.15.37:0/277919793 learned_addr learned my addr 172.21.15.37:0/277919793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a18bb640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c103b20 0x7f189c101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a20bc640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 0x7f189c100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 -- 172.21.15.37:0/277919793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 msgr2=0x7f189c100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 0x7f189c100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 -- 172.21.15.37:0/277919793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c103b20 msgr2=0x7f189c101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c103b20 0x7f189c101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 -- 172.21.15.37:0/277919793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c1a6bd0 con 0x7f189c10c5d0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a28bd640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f189400d720 tx=0x7f189400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18937fe640 1 -- 172.21.15.37:0/277919793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894017070 con 0x7f189c10c5d0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18937fe640 1 -- 172.21.15.37:0/277919793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f189400de30 con 0x7f189c10c5d0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a6ec0 con 0x7f189c10c5d0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.824+0000 7f18937fe640 1 -- 172.21.15.37:0/277919793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894004d00 con 0x7f189c10c5d0 2024-04-25T21:25:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.825+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a7390 con 0x7f189c10c5d0 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.827+0000 7f18937fe640 1 -- 172.21.15.37:0/277919793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1894004ea0 con 0x7f189c10c5d0 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.827+0000 7f18a703a740 1 Processor -- start 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 -- start start 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cbea824970 con 0x55cbea8bf620 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cbea7c14a0 con 0x55cbea8ca800 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a703a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cbea820930 con 0x55cbea8bff40 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a20bc640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18937fe640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 0x7f18880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8bfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52220/0 (socket says 172.21.15.37:52220) 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a18bb640 1 -- 172.21.15.37:0/2310226818 learned_addr learned my addr 172.21.15.37:0/2310226818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a18bb640 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18a28bd640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 0x7f18880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.828+0000 7f18937fe640 1 -- 172.21.15.37:0/277919793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f18940d00b0 con 0x7f189c10c5d0 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 -- 172.21.15.37:0/2310226818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 msgr2=0x55cbea8ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 -- 172.21.15.37:0/2310226818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 msgr2=0x55cbea8bfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8bfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 -- 172.21.15.37:0/2310226818 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbea537210 con 0x55cbea8bff40 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a28bd640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 0x7f18880a6280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18940cd8b0 tx=0x7f189406aa00 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f18a20bc640 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8ca2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f189800adb0 tx=0x7f189800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a9e82654ec4ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f1890bfe640 1 -- 172.21.15.37:0/2310226818 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898003970 con 0x55cbea8bff40 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f1890bfe640 1 -- 172.21.15.37:0/2310226818 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1898004020 con 0x55cbea8bff40 2024-04-25T21:25:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.829+0000 7f1890bfe640 1 -- 172.21.15.37:0/2310226818 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18980119d0 con 0x55cbea8bff40 2024-04-25T21:25:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- 172.21.15.37:0/2310226818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 msgr2=0x55cbea8ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8ca2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f189800adb0 tx=0x7f189800f730 comp rx=0 tx=0).stop 2024-04-25T21:25:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- 172.21.15.37:0/2310226818 shutdown_connections 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8ca2c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 --2- 172.21.15.37:0/2310226818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- 172.21.15.37:0/2310226818 >> 172.21.15.37:0/2310226818 conn(0x55cbea8bbb30 msgr2=0x55cbea8bbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- 172.21.15.37:0/2310226818 shutdown_connections 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- 172.21.15.37:0/2310226818 wait complete. 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 -- start start 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.830+0000 7f18a703a740 1 Processor -- start 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bf620 0x55cbea8ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cbea8cd430 con 0x55cbea8bff40 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55cbea7bf0b0 con 0x55cbea8ca800 2024-04-25T21:25:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55cbea8cd5b0 con 0x55cbea8bf620 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 10 client.0 _ll_register_callbacks cb 0x55cbea7acd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52234/0 (socket says 172.21.15.37:52234) 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 -- 172.21.15.37:0/1245587502 learned_addr learned my addr 172.21.15.37:0/1245587502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a18bb640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a18bb640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bf620 0x55cbea8ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 -- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bf620 msgr2=0x55cbea8ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bf620 0x55cbea8ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 -- 172.21.15.37:0/1245587502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 msgr2=0x55cbea8c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 -- 172.21.15.37:0/1245587502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbea603620 con 0x55cbea8bff40 2024-04-25T21:25:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a18bb640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a20bc640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1898006010 tx=0x7f1898004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c0e446b69359ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f1885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898027060 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f1898004c50 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.831+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1898004020 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1898003970 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 10 client.17313 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbea8d93a0 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1898004020 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1898011c70 con 0x55cbea8bff40 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 10 client.17313 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 20 client.17313 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea8dcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 4 client.17313 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbea832390 con 0x55cbea8da800 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.832+0000 7f18a18bb640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea8dcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.833+0000 7f18a18bb640 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea8dcbf0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc399b15830636b0 server_cookie=e8c7d18889ff2bde in_seq=0 out_seq=0 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.833+0000 7f1885ffb640 10 client.17313 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f1885ffb640 1 -- 172.21.15.37:0/1245587502 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbea832390 con 0x55cbea8da800 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f1885ffb640 10 client.17313 handle_command_reply: tid=0 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 client.17313 shutdown 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 msgr2=0x55cbea8c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1898006010 tx=0x7f1898004400 comp rx=0 tx=0).stop 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 shutdown_connections 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea8dcbf0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8ca800 0x55cbea8c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bff40 0x55cbea8c7460 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f1885ffb640 0 client.17313 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 --2- 172.21.15.37:0/1245587502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bf620 0x55cbea8ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 >> 172.21.15.37:0/1245587502 conn(0x55cbea8bbb30 msgr2=0x55cbea8c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 shutdown_connections 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 1 -- 172.21.15.37:0/1245587502 wait complete. 2024-04-25T21:25:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.834+0000 7f18a703a740 20 client.17313 trim_cache size 0 max 16384 2024-04-25T21:25:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 Processor -- start 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 -- start start 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bf620 0x55cbea8d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1afd10 con 0x55cbea8ca800 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b1850 con 0x55cbea8bf620 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.849+0000 7f18a703a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b03e0 con 0x55cbea8bff40 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52248/0 (socket says 172.21.15.37:52248) 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 -- 172.21.15.37:0/2656833830 learned_addr learned my addr 172.21.15.37:0/2656833830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a20bc640 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bf620 0x55cbea8d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 -- 172.21.15.37:0/2656833830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 msgr2=0x55cbea8c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 -- 172.21.15.37:0/2656833830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bf620 msgr2=0x55cbea8d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bf620 0x55cbea8d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 -- 172.21.15.37:0/2656833830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbea8ccbf0 con 0x55cbea8ca800 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f18a18bb640 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f188c009230 tx=0x7f188c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11d4fd1d2549da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f1887fff640 1 -- 172.21.15.37:0/2656833830 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c04f070 con 0x55cbea8ca800 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f1887fff640 1 -- 172.21.15.37:0/2656833830 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0054f0 con 0x55cbea8ca800 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.850+0000 7f1887fff640 1 -- 172.21.15.37:0/2656833830 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c035040 con 0x55cbea8ca800 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- 172.21.15.37:0/2656833830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 msgr2=0x55cbea8c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f188c009230 tx=0x7f188c02a480 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- 172.21.15.37:0/2656833830 shutdown_connections 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8ca800 0x55cbea8c9ca0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8bff40 0x55cbea8c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- 172.21.15.37:0/2656833830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bf620 0x55cbea8d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- 172.21.15.37:0/2656833830 >> 172.21.15.37:0/2656833830 conn(0x55cbea8bbb30 msgr2=0x55cbea8c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- 172.21.15.37:0/2656833830 shutdown_connections 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- 172.21.15.37:0/2656833830 wait complete. 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- start start 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 Processor -- start 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 0x55cbea8ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8ca800 0x55cbea8cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b1180 con 0x55cbea8bf620 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1b0ab0 con 0x55cbea8bff40 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.851+0000 7f18a703a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f189c1add10 con 0x55cbea8ca800 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a703a740 10 client.0 _ll_register_callbacks cb 0x7f189800ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52256/0 (socket says 172.21.15.37:52256) 2024-04-25T21:25:09.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 -- 172.21.15.37:0/3488809434 learned_addr learned my addr 172.21.15.37:0/3488809434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8ca800 0x55cbea8cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a20bc640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 0x55cbea8ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 -- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8ca800 msgr2=0x55cbea8cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8ca800 0x55cbea8cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 -- 172.21.15.37:0/3488809434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 msgr2=0x55cbea8ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 0x55cbea8ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 -- 172.21.15.37:0/3488809434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbea93fa30 con 0x55cbea8bf620 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a20bc640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 0x55cbea8ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a18bb640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f188c03f040 tx=0x7f188c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d432c977d3ca0ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f1885ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c035070 con 0x55cbea8bf620 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f188c04a6d0 con 0x55cbea8bf620 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f188c0054f0 con 0x55cbea8bf620 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f188c04f070 con 0x55cbea8bf620 2024-04-25T21:25:09.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a703a740 10 client.17319 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.852+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cbea940ca0 con 0x55cbea8bf620 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f188c0054f0 con 0x55cbea8bf620 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f188c0320c0 con 0x55cbea8bf620 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 10 client.17319 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 20 client.17319 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea9432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 4 client.17319 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbea82c200 con 0x55cbea8da800 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.853+0000 7f18a20bc640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea9432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.854+0000 7f18a20bc640 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea9432e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0c43b2f6308b37 server_cookie=4e91ee8be341efa9 in_seq=0 out_seq=0 2024-04-25T21:25:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.854+0000 7f1885ffb640 10 client.17319 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f1885ffb640 1 -- 172.21.15.37:0/3488809434 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbea82c200 con 0x55cbea8da800 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f1885ffb640 10 client.17319 handle_command_reply: tid=0 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 client.17319 shutdown 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 msgr2=0x55cbea8ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f188c03f040 tx=0x7f188c009520 comp rx=0 tx=0).stop 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 shutdown_connections 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55cbea8da800 0x55cbea9432e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cbea8ca800 0x55cbea8cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55cbea8bff40 0x55cbea8ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f1885ffb640 0 client.17319 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 --2- 172.21.15.37:0/3488809434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cbea8bf620 0x55cbea8ce1f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 >> 172.21.15.37:0/3488809434 conn(0x55cbea8bbb30 msgr2=0x55cbea8d52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 shutdown_connections 2024-04-25T21:25:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 1 -- 172.21.15.37:0/3488809434 wait complete. 2024-04-25T21:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.869+0000 7f18a703a740 20 client.17319 trim_cache size 0 max 16384 2024-04-25T21:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:25:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 msgr2=0x7f18880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 0x7f18880a6280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18940cd8b0 tx=0x7f189406aa00 comp rx=0 tx=0).stop 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 msgr2=0x7f189c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f189400d720 tx=0x7f189400dbe0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 shutdown_connections 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f189c10c5d0 0x7f189c1018d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f189c103b20 0x7f189c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f18880a3dc0 0x7f18880a6280 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 --2- 172.21.15.37:0/277919793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f189c103200 0x7f189c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 >> 172.21.15.37:0/277919793 conn(0x7f189c0771d0 msgr2=0x7f189c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 shutdown_connections 2024-04-25T21:25:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:09.870+0000 7f18a4320640 1 -- 172.21.15.37:0/277919793 wait complete. 2024-04-25T21:25:10.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:09 smithi177 ceph-mon[33043]: pgmap v525: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:25:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:09 smithi098 ceph-mon[32569]: pgmap v525: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:25:10.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:09 smithi037 ceph-mon[29195]: pgmap v525: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 95 KiB/s wr, 34 op/s 2024-04-25T21:25:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:11 smithi177 ceph-mon[33043]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 41 op/s 2024-04-25T21:25:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:11 smithi098 ceph-mon[32569]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 41 op/s 2024-04-25T21:25:12.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:11 smithi037 ceph-mon[29195]: pgmap v526: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 115 KiB/s wr, 41 op/s 2024-04-25T21:25:14.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:13 smithi177 ceph-mon[33043]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:25:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:13 smithi098 ceph-mon[32569]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:25:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:13 smithi037 ceph-mon[29195]: pgmap v527: 97 pgs: 97 active+clean; 4.1 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 99 KiB/s wr, 35 op/s 2024-04-25T21:25:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:15 smithi177 ceph-mon[33043]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:25:16.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:15 smithi098 ceph-mon[32569]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:25:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:15 smithi037 ceph-mon[29195]: pgmap v528: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-25T21:25:18.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:17 smithi177 ceph-mon[33043]: pgmap v529: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:25:18.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:17 smithi098 ceph-mon[32569]: pgmap v529: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:25:18.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:17 smithi037 ceph-mon[29195]: pgmap v529: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-25T21:25:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:19 smithi177 ceph-mon[33043]: pgmap v530: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:25:20.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:19 smithi098 ceph-mon[32569]: pgmap v530: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:25:20.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:19 smithi037 ceph-mon[29195]: pgmap v530: 97 pgs: 97 active+clean; 4.3 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-25T21:25:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:21 smithi177 ceph-mon[33043]: pgmap v531: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T21:25:22.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:21 smithi098 ceph-mon[32569]: pgmap v531: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T21:25:22.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:21 smithi037 ceph-mon[29195]: pgmap v531: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-25T21:25:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 -- 172.21.15.37:0/272987230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f4101aa0 msgr2=0x7f73f4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/272987230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f4101aa0 0x7f73f4105a90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009900 tx=0x7f73e402a410 comp rx=0 tx=0).stop 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 -- 172.21.15.37:0/272987230 shutdown_connections 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/272987230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 0x7f73f410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/272987230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f4101aa0 0x7f73f4105a90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/272987230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 0x7f73f41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 -- 172.21.15.37:0/272987230 >> 172.21.15.37:0/272987230 conn(0x7f73f4077fd0 msgr2=0x7f73f40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.970+0000 7f73fbc7b640 1 -- 172.21.15.37:0/272987230 shutdown_connections 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 -- 172.21.15.37:0/272987230 wait complete. 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 Processor -- start 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 -- start start 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 0x7f73f41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 0x7f73f4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4111100 con 0x7f73f419cf80 2024-04-25T21:25:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4110f80 con 0x7f73f4106170 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fbc7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73f4111280 con 0x7f73f41010f0 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.971+0000 7f73fa218640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38276/0 (socket says 172.21.15.37:38276) 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 -- 172.21.15.37:0/385037426 learned_addr learned my addr 172.21.15.37:0/385037426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73f9216640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 0x7f73f4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73f9a17640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 0x7f73f41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 -- 172.21.15.37:0/385037426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 msgr2=0x7f73f41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 0x7f73f41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 -- 172.21.15.37:0/385037426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 msgr2=0x7f73f4198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 0x7f73f4198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 -- 172.21.15.37:0/385037426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f419da70 con 0x7f73f419cf80 2024-04-25T21:25:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fa218640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00eaa0 tx=0x7f73ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73eaffd640 1 -- 172.21.15.37:0/385037426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec00cdb0 con 0x7f73f419cf80 2024-04-25T21:25:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73eaffd640 1 -- 172.21.15.37:0/385037426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73ec005020 con 0x7f73f419cf80 2024-04-25T21:25:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f419dd60 con 0x7f73f419cf80 2024-04-25T21:25:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73eaffd640 1 -- 172.21.15.37:0/385037426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0116d0 con 0x7f73f419cf80 2024-04-25T21:25:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.972+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f41a5520 con 0x7f73f419cf80 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.975+0000 7f73eaffd640 1 -- 172.21.15.37:0/385037426 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73ec0051d0 con 0x7f73f419cf80 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.975+0000 7f73fe98a740 1 Processor -- start 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.975+0000 7f73fe98a740 1 -- start start 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b12e800 0x558b1b130bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b1b088970 con 0x558b1b123620 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558b1b0254a0 con 0x558b1b12e800 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fe98a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558b1b084930 con 0x558b1b123f40 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73eaffd640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 0x7f73e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fa218640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 0x7f73e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38288/0 (socket says 172.21.15.37:38288) 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 -- 172.21.15.37:0/1142226002 learned_addr learned my addr 172.21.15.37:0/1142226002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73eaffd640 1 -- 172.21.15.37:0/385037426 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f73ec0d0850 con 0x7f73f419cf80 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9a17640 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b12e800 0x558b1b130bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 -- 172.21.15.37:0/1142226002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 msgr2=0x558b1b12e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 -- 172.21.15.37:0/1142226002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b12e800 msgr2=0x558b1b130bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b12e800 0x558b1b130bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 -- 172.21.15.37:0/1142226002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1ad9b210 con 0x558b1b123620 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73f9216640 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006010 tx=0x7f73e4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2386d8c2461b697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.976+0000 7f73fa218640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 0x7f73e00a6300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00cfd0 tx=0x7f73ec005890 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73dffff640 1 -- 172.21.15.37:0/1142226002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4044020 con 0x558b1b123620 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73dffff640 1 -- 172.21.15.37:0/1142226002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4004750 con 0x558b1b123620 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73dffff640 1 -- 172.21.15.37:0/1142226002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4035040 con 0x558b1b123620 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 -- 172.21.15.37:0/1142226002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 msgr2=0x558b1b123a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f73e4006010 tx=0x7f73e4005e50 comp rx=0 tx=0).stop 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 -- 172.21.15.37:0/1142226002 shutdown_connections 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b12e800 0x558b1b130bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.977+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1142226002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b123620 0x558b1b123a00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.978+0000 7f73fe98a740 1 -- 172.21.15.37:0/1142226002 >> 172.21.15.37:0/1142226002 conn(0x558b1b11fb30 msgr2=0x558b1b11ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.978+0000 7f73fe98a740 1 -- 172.21.15.37:0/1142226002 shutdown_connections 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.978+0000 7f73fe98a740 1 -- 172.21.15.37:0/1142226002 wait complete. 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.978+0000 7f73fe98a740 1 -- start start 2024-04-25T21:25:22.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.978+0000 7f73fe98a740 1 Processor -- start 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b12e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558b1b1313d0 con 0x558b1b12e800 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558b1b0230b0 con 0x558b1b123620 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x558b1b131550 con 0x558b1b123f40 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 10 client.0 _ll_register_callbacks cb 0x558b1b010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:22.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38294/0 (socket says 172.21.15.37:38294) 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 -- 172.21.15.37:0/4281950760 learned_addr learned my addr 172.21.15.37:0/4281950760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9a17640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b12e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 -- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 msgr2=0x558b1b12b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 -- 172.21.15.37:0/4281950760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 msgr2=0x558b1b12e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b12e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 -- 172.21.15.37:0/4281950760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1ae67620 con 0x558b1b12e800 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73f9216640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a450 tx=0x7f73e4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d0ad346cf497bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4051060 con 0x558b1b12e800 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f73e4005240 con 0x558b1b12e800 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.979+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e4004750 con 0x558b1b12e800 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73e4035030 con 0x558b1b12e800 2024-04-25T21:25:22.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 10 client.17331 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b1b13d300 con 0x558b1b12e800 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4004750 con 0x558b1b12e800 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f73e40314a0 con 0x558b1b12e800 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 10 client.17331 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 10 client.17331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 20 client.17331 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b13e650 0x558b1b140a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 4 client.17331 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b1b096390 con 0x558b1b13e650 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.980+0000 7f73f9a17640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b13e650 0x558b1b140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.981+0000 7f73f9a17640 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b13e650 0x558b1b140a40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5fc7a4b4da9d89 server_cookie=a8dad00047b2da64 in_seq=0 out_seq=0 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.981+0000 7f73dd7fa640 10 client.17331 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.981+0000 7f73dd7fa640 1 -- 172.21.15.37:0/4281950760 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b1b096390 con 0x558b1b13e650 2024-04-25T21:25:22.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.981+0000 7f73dd7fa640 10 client.17331 handle_command_reply: tid=0 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 client.17331 shutdown 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 msgr2=0x558b1b12b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a450 tx=0x7f73e4004610 comp rx=0 tx=0).stop 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 shutdown_connections 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b13e650 0x558b1b140a40 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12b9b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b12b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 --2- 172.21.15.37:0/4281950760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b12e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73dd7fa640 0 client.17331 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 >> 172.21.15.37:0/4281950760 conn(0x558b1b11fb30 msgr2=0x558b1b1258b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 shutdown_connections 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 1 -- 172.21.15.37:0/4281950760 wait complete. 2024-04-25T21:25:22.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.982+0000 7f73fe98a740 20 client.17331 trim_cache size 0 max 16384 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 Processor -- start 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 -- start start 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123620 0x558b1b13a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123f40 0x558b1b126850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41ad990 con 0x558b1b12e800 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41af4d0 con 0x558b1b123f40 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73fe98a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41ae060 con 0x558b1b123620 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38300/0 (socket says 172.21.15.37:38300) 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73f9216640 1 -- 172.21.15.37:0/738369612 learned_addr learned my addr 172.21.15.37:0/738369612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73f9216640 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123620 0x558b1b13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.997+0000 7f73f9a17640 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123f40 0x558b1b126850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 -- 172.21.15.37:0/738369612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123620 msgr2=0x558b1b13a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123620 0x558b1b13a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 -- 172.21.15.37:0/738369612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123f40 msgr2=0x558b1b126850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123f40 0x558b1b126850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 -- 172.21.15.37:0/738369612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1b120470 con 0x558b1b12e800 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73f9216640 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a450 tx=0x7f73e40041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae34cd9be07d02fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73df7fe640 1 -- 172.21.15.37:0/738369612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4044070 con 0x558b1b12e800 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73df7fe640 1 -- 172.21.15.37:0/738369612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4005850 con 0x558b1b12e800 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73df7fe640 1 -- 172.21.15.37:0/738369612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40036a0 con 0x558b1b12e800 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73fe98a740 1 -- 172.21.15.37:0/738369612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 msgr2=0x558b1b12dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.998+0000 7f73fe98a740 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a450 tx=0x7f73e40041a0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- 172.21.15.37:0/738369612 shutdown_connections 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b12dc90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123f40 0x558b1b126850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- 172.21.15.37:0/738369612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123620 0x558b1b13a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- 172.21.15.37:0/738369612 >> 172.21.15.37:0/738369612 conn(0x558b1b11fb30 msgr2=0x558b1b126140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- 172.21.15.37:0/738369612 shutdown_connections 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- 172.21.15.37:0/738369612 wait complete. 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- start start 2024-04-25T21:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 Processor -- start 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b136bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b131f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41aee00 con 0x558b1b12e800 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41ae730 con 0x558b1b123620 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:22.999+0000 7f73fe98a740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f73f41ab990 con 0x558b1b123f40 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73fe98a740 10 client.0 _ll_register_callbacks cb 0x7f73e40035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38308/0 (socket says 172.21.15.37:38308) 2024-04-25T21:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 -- 172.21.15.37:0/1067097980 learned_addr learned my addr 172.21.15.37:0/1067097980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b136bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9a17640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b131f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 -- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 msgr2=0x558b1b131f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b131f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 -- 172.21.15.37:0/1067097980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 msgr2=0x558b1b136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b136bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 -- 172.21.15.37:0/1067097980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1b132cd0 con 0x558b1b12e800 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73f9216640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a930 tx=0x7f73e4031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec8a39431b6c541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4064060 con 0x558b1b12e800 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f73e4005110 con 0x558b1b12e800 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e4005850 con 0x558b1b12e800 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f73e4044040 con 0x558b1b12e800 2024-04-25T21:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73fe98a740 10 client.17337 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.000+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558b1b1a4920 con 0x558b1b12e800 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4005850 con 0x558b1b12e800 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f73e4055460 con 0x558b1b12e800 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 10 client.17337 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 20 client.17337 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b1a4ed0 0x558b1b1a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 4 client.17337 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b1b090200 con 0x558b1b1a4ed0 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.001+0000 7f73f9a17640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b1a4ed0 0x558b1b1a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.002+0000 7f73dd7fa640 10 client.17337 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:23.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.002+0000 7f73f9a17640 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b1a4ed0 0x558b1b1a72c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e5dcc94a6f636f server_cookie=17af6c411d1ae7bc in_seq=0 out_seq=0 2024-04-25T21:25:23.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.014+0000 7f73dd7fa640 1 -- 172.21.15.37:0/1067097980 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b1b090200 con 0x558b1b1a4ed0 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.014+0000 7f73dd7fa640 10 client.17337 handle_command_reply: tid=0 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.014+0000 7f73fe98a740 1 client.17337 shutdown 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.014+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 msgr2=0x558b1b1324a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.014+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f73e402a930 tx=0x7f73e4031770 comp rx=0 tx=0).stop 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 shutdown_connections 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x558b1b1a4ed0 0x558b1b1a72c0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558b1b12e800 0x558b1b1324a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558b1b123f40 0x558b1b131f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73dd7fa640 0 client.17337 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 --2- 172.21.15.37:0/1067097980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558b1b123620 0x558b1b136bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 >> 172.21.15.37:0/1067097980 conn(0x558b1b11fb30 msgr2=0x558b1b11ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 shutdown_connections 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 1 -- 172.21.15.37:0/1067097980 wait complete. 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fe98a740 20 client.17337 trim_cache size 0 max 16384 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:25:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 msgr2=0x7f73e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 0x7f73e00a6300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00cfd0 tx=0x7f73ec005890 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 msgr2=0x7f73f419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00eaa0 tx=0x7f73ec00ef60 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 shutdown_connections 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73f419cf80 0x7f73f419d430 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73f4106170 0x7f73f4198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f73e00a3e40 0x7f73e00a6300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 --2- 172.21.15.37:0/385037426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73f41010f0 0x7f73f41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 >> 172.21.15.37:0/385037426 conn(0x7f73f4077fd0 msgr2=0x7f73f4104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 shutdown_connections 2024-04-25T21:25:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:23.015+0000 7f73fbc7b640 1 -- 172.21.15.37:0/385037426 wait complete. 2024-04-25T21:25:24.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:23 smithi177 ceph-mon[33043]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:25:24.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:23 smithi098 ceph-mon[32569]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:25:24.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:23 smithi037 ceph-mon[29195]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-25T21:25:26.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:25 smithi177 ceph-mon[33043]: pgmap v533: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-25T21:25:26.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:25 smithi098 ceph-mon[32569]: pgmap v533: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-25T21:25:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:25 smithi037 ceph-mon[29195]: pgmap v533: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-25T21:25:28.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:27 smithi177 ceph-mon[33043]: pgmap v534: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:28.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:27 smithi098 ceph-mon[32569]: pgmap v534: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:28.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:27 smithi037 ceph-mon[29195]: pgmap v534: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:30.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:29 smithi177 ceph-mon[33043]: pgmap v535: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:30.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:29 smithi098 ceph-mon[32569]: pgmap v535: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:30.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:29 smithi037 ceph-mon[29195]: pgmap v535: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-25T21:25:32.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:31 smithi177 ceph-mon[33043]: pgmap v536: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:25:32.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:31 smithi098 ceph-mon[32569]: pgmap v536: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:25:32.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:31 smithi037 ceph-mon[29195]: pgmap v536: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-25T21:25:32.909 INFO:tasks.workunit.client.0.smithi037.stdout:561/600: open fd = 3 2024-04-25T21:25:32.909 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.909 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.909 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:562/600: open fd = 3 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:563/600: open fd = 3 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:564/600: open fd = 3 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:565/600: open fd = 3 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.910 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:566/600: open fd = 3 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:567/600: open fd = 3 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:568/600: open fd = 3 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:569/600: open fd = 3 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.911 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:570/600: open fd = 3 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:571/600: open fd = 3 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:572/600: open fd = 3 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:573/600: open fd = 3 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.912 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:574/600: open fd = 3 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:575/600: open fd = 3 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:576/600: open fd = 3 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:577/600: open fd = 3 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.913 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:578/600: open fd = 3 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:579/600: open fd = 3 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:580/600: open fd = 3 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.914 INFO:tasks.workunit.client.0.smithi037.stdout:581/600: open fd = 3 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:582/600: open fd = 3 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:583/600: open fd = 3 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:584/600: open fd = 3 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.915 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:585/600: open fd = 3 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:586/600: open fd = 3 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:587/600: open fd = 3 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:588/600: open fd = 3 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.916 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:589/600: open fd = 3 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:590/600: open fd = 3 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:591/600: open fd = 3 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.917 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:592/600: open fd = 3 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:593/600: open fd = 3 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:594/600: open fd = 3 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.918 INFO:tasks.workunit.client.0.smithi037.stdout:595/600: open fd = 3 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:596/600: open fd = 3 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:597/600: open fd = 3 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:598/600: open fd = 3 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.919 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:599/600: open fd = 3 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:600/600: open fd = 3 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:write ret = 32 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stdout:pread ret = 64 2024-04-25T21:25:32.920 INFO:tasks.workunit.client.0.smithi037.stderr:+ echo OK 2024-04-25T21:25:32.921 INFO:tasks.workunit.client.0.smithi037.stdout:OK 2024-04-25T21:25:32.921 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T21:25:32.922 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T21:25:33.009 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-25T21:25:33.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T21:25:33.323 DEBUG:teuthology.parallel:result is None 2024-04-25T21:25:33.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:25:33.360 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:25:33.360 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T21:25:33.372 INFO:tasks.quiescer:joining Quiescers 2024-04-25T21:25:33.372 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T21:25:33.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:25:33.646 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:34.035 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:33 smithi037 ceph-mon[29195]: pgmap v537: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:25:34.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:33 smithi177 ceph-mon[33043]: pgmap v537: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:25:34.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:33 smithi098 ceph-mon[32569]: pgmap v537: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-25T21:25:34.303 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T21:25:34.303 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-04-25T21:25:34.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:25:35.401 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 -- 172.21.15.37:0/3143686716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea80108a30 msgr2=0x7fea80108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 --2- 172.21.15.37:0/3143686716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea80108a30 0x7fea80108e10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009990 tx=0x7fea7c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 -- 172.21.15.37:0/3143686716 shutdown_connections 2024-04-25T21:25:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 --2- 172.21.15.37:0/3143686716 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea8006bb10 0x7fea8010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 --2- 172.21.15.37:0/3143686716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea80108a30 0x7fea80108e10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 --2- 172.21.15.37:0/3143686716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006b170 0x7fea8006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 -- 172.21.15.37:0/3143686716 >> 172.21.15.37:0/3143686716 conn(0x7fea800788f0 msgr2=0x7fea80078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 -- 172.21.15.37:0/3143686716 shutdown_connections 2024-04-25T21:25:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.621+0000 7fea876d1700 1 -- 172.21.15.37:0/3143686716 wait complete. 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 Processor -- start 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 -- start start 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006bb10 0x7fea80199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 0x7fea8019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea80112bd0 con 0x7fea8006b170 2024-04-25T21:25:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea80112a50 con 0x7fea80108a30 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea876d1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fea80112d50 con 0x7fea8006bb10 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea8546d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea8546d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58530/0 (socket says 172.21.15.37:58530) 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea8546d700 1 -- 172.21.15.37:0/3557542794 learned_addr learned my addr 172.21.15.37:0/3557542794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.622+0000 7fea84c6c700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006bb10 0x7fea80199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea85c6e700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 0x7fea8019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 -- 172.21.15.37:0/3557542794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006bb10 msgr2=0x7fea80199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006bb10 0x7fea80199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 -- 172.21.15.37:0/3557542794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 msgr2=0x7fea8019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 0x7fea8019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 -- 172.21.15.37:0/3557542794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8019e110 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea85c6e700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 0x7fea8019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea8546d700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c027070 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8019e3a0 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7c003f20 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c022cf0 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.623+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea801a5c80 con 0x7fea8006b170 2024-04-25T21:25:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.624+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea80069730 con 0x7fea8006b170 2024-04-25T21:25:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.625+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea7c02a070 con 0x7fea8006b170 2024-04-25T21:25:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.626+0000 7fea767fc700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 0x7fea6c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.626+0000 7fea84c6c700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 0x7fea6c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.626+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fea7c0d15b0 con 0x7fea8006b170 2024-04-25T21:25:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.626+0000 7fea84c6c700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 0x7fea6c0a6270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea80078f60 tx=0x7fea7000b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.631+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea7c013070 con 0x7fea8006b170 2024-04-25T21:25:35.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:35 smithi037 ceph-mon[29195]: pgmap v538: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:25:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.893+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea8002df80 con 0x7fea8006b170 2024-04-25T21:25:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.894+0000 7fea767fc700 1 -- 172.21.15.37:0/3557542794 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fea7c09dce0 con 0x7fea8006b170 2024-04-25T21:25:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:25:35.894 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:25:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 msgr2=0x7fea6c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 0x7fea6c0a6270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fea80078f60 tx=0x7fea7000b3c0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 msgr2=0x7fea80199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fea7c000c00 tx=0x7fea7c0047b0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 shutdown_connections 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fea6c0a3db0 0x7fea6c0a6270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea80108a30 0x7fea8019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea8006b170 0x7fea80199100 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 --2- 172.21.15.37:0/3557542794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea8006bb10 0x7fea80199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 >> 172.21.15.37:0/3557542794 conn(0x7fea800788f0 msgr2=0x7fea800ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 shutdown_connections 2024-04-25T21:25:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:35.896+0000 7fea876d1700 1 -- 172.21.15.37:0/3557542794 wait complete. 2024-04-25T21:25:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:25:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:35 smithi177 ceph-mon[33043]: pgmap v538: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- 172.21.15.37:0/2643079487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed741031e0 msgr2=0x7fed741035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2643079487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed741031e0 0x7fed741035c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fed64009860 tx=0x7fed6402a460 comp rx=0 tx=0).stop 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- 172.21.15.37:0/2643079487 shutdown_connections 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2643079487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed7410c520 0x7fed7410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2643079487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 0x7fed7410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2643079487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed741031e0 0x7fed741035c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- 172.21.15.37:0/2643079487 >> 172.21.15.37:0/2643079487 conn(0x7fed74077fd0 msgr2=0x7fed740783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- 172.21.15.37:0/2643079487 shutdown_connections 2024-04-25T21:25:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- 172.21.15.37:0/2643079487 wait complete. 2024-04-25T21:25:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 Processor -- start 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.107+0000 7fed79aa9640 1 -- start start 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed741031e0 0x7fed74198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 0x7fed74198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed741121e0 con 0x7fed7410c520 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed74112060 con 0x7fed741031e0 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed79aa9640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed74112360 con 0x7fed74103b00 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58554/0 (socket says 172.21.15.37:58554) 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 -- 172.21.15.37:0/2206836457 learned_addr learned my addr 172.21.15.37:0/2206836457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed72ffd640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 0x7fed74198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed737fe640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed741031e0 0x7fed74198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 -- 172.21.15.37:0/2206836457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 msgr2=0x7fed74198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 0x7fed74198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 -- 172.21.15.37:0/2206836457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed741031e0 msgr2=0x7fed74198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed741031e0 0x7fed74198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 -- 172.21.15.37:0/2206836457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7419d700 con 0x7fed7410c520 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed73fff640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00eb10 tx=0x7fed6c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.108+0000 7fed70ff9640 1 -- 172.21.15.37:0/2206836457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00ce20 con 0x7fed7410c520 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.109+0000 7fed70ff9640 1 -- 172.21.15.37:0/2206836457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6c005020 con 0x7fed7410c520 2024-04-25T21:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.109+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7419d9f0 con 0x7fed7410c520 2024-04-25T21:25:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.109+0000 7fed70ff9640 1 -- 172.21.15.37:0/2206836457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c010690 con 0x7fed7410c520 2024-04-25T21:25:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.109+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed741a5280 con 0x7fed7410c520 2024-04-25T21:25:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.111+0000 7fed70ff9640 1 -- 172.21.15.37:0/2206836457 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed6c005510 con 0x7fed7410c520 2024-04-25T21:25:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.111+0000 7fed70ff9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 0x7fed5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.111+0000 7fed737fe640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 0x7fed5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed70ff9640 1 -- 172.21.15.37:0/2206836457 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fed6c0d0ce0 con 0x7fed7410c520 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed737fe640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 0x7fed5c0a6280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed6402aa50 tx=0x7fed640058b0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 Processor -- start 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 -- start start 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 0x55ba83216a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 0x55ba83223bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ba8317b970 con 0x55ba83216f40 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba831184a0 con 0x55ba83216620 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.112+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ba83177930 con 0x55ba83221800 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58558/0 (socket says 172.21.15.37:58558) 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 -- 172.21.15.37:0/3462343561 learned_addr learned my addr 172.21.15.37:0/3462343561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed737fe640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 0x55ba83216a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed73fff640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 0x55ba83223bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 -- 172.21.15.37:0/3462343561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 msgr2=0x55ba83223bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 0x55ba83223bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 -- 172.21.15.37:0/3462343561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 msgr2=0x55ba83216a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 0x55ba83216a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 -- 172.21.15.37:0/3462343561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba82e8e210 con 0x55ba83216f40 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed73fff640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 0x55ba83223bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed737fe640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 0x55ba83216a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ca0 tx=0x7fed68011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e34798cd8c4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed59ffb640 1 -- 172.21.15.37:0/3462343561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68004230 con 0x55ba83216f40 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed59ffb640 1 -- 172.21.15.37:0/3462343561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed680043d0 con 0x55ba83216f40 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.113+0000 7fed59ffb640 1 -- 172.21.15.37:0/3462343561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed680196d0 con 0x55ba83216f40 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3462343561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 msgr2=0x55ba832212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fed68009ca0 tx=0x7fed68011810 comp rx=0 tx=0).stop 2024-04-25T21:25:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3462343561 shutdown_connections 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83221800 0x55ba83223bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba832212c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3462343561 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83216620 0x55ba83216a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3462343561 >> 172.21.15.37:0/3462343561 conn(0x55ba83212b30 msgr2=0x55ba83212f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3462343561 shutdown_connections 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3462343561 wait complete. 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.114+0000 7fed7c7bd740 1 -- start start 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 Processor -- start 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba8321e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba8321e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ba83224430 con 0x55ba83216620 2024-04-25T21:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ba831160b0 con 0x55ba83221800 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x55ba832245b0 con 0x55ba83216f40 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed7c7bd740 10 client.0 _ll_register_callbacks cb 0x55ba83103d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58574/0 (socket says 172.21.15.37:58574) 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 -- 172.21.15.37:0/962272472 learned_addr learned my addr 172.21.15.37:0/962272472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed73fff640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba8321e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed72ffd640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba8321e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 -- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 msgr2=0x55ba8321e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba8321e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 -- 172.21.15.37:0/962272472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 msgr2=0x55ba8321e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba8321e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed737fe640 1 -- 172.21.15.37:0/962272472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba82f5a620 con 0x55ba83216620 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.115+0000 7fed72ffd640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba8321e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed737fe640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fed6402a980 tx=0x7fed6402a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f115efb7ec3796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed640048c0 con 0x55ba83216620 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fed64004a60 con 0x55ba83216620 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed680043d0 con 0x55ba83216620 2024-04-25T21:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed64004c60 con 0x55ba83216620 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 10 client.17355 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba83230270 con 0x55ba83216620 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed680043d0 con 0x55ba83216620 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed6404f070 con 0x55ba83216620 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 10 client.17355 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 20 client.17355 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83231770 0x55ba83233be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 4 client.17355 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.116+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba83189390 con 0x55ba83231770 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.117+0000 7fed72ffd640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83231770 0x55ba83233be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.117+0000 7fed52ffd640 10 client.17355 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.117+0000 7fed72ffd640 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83231770 0x55ba83233be0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9810822d6765512d server_cookie=b526d379bb735646 in_seq=0 out_seq=0 2024-04-25T21:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed52ffd640 1 -- 172.21.15.37:0/962272472 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba83189390 con 0x55ba83231770 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed52ffd640 10 client.17355 handle_command_reply: tid=0 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 client.17355 shutdown 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 msgr2=0x55ba83221100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fed6402a980 tx=0x7fed6402a460 comp rx=0 tx=0).stop 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 shutdown_connections 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83231770 0x55ba83233be0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba8321e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba8321e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed52ffd640 0 client.17355 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/962272472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83221100 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.118+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 >> 172.21.15.37:0/962272472 conn(0x55ba83212b30 msgr2=0x55ba832188b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.119+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 shutdown_connections 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.119+0000 7fed7c7bd740 1 -- 172.21.15.37:0/962272472 wait complete. 2024-04-25T21:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.119+0000 7fed7c7bd740 20 client.17355 trim_cache size 0 max 16384 2024-04-25T21:25:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 Processor -- start 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 -- start start 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba832394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83239a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed741adb80 con 0x55ba83216620 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed741af6c0 con 0x55ba83221800 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.150+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed741ae250 con 0x55ba83216f40 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58576/0 (socket says 172.21.15.37:58576) 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 -- 172.21.15.37:0/1002755633 learned_addr learned my addr 172.21.15.37:0/1002755633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed72ffd640 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba832394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed73fff640 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83239a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 -- 172.21.15.37:0/1002755633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 msgr2=0x55ba832394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba832394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 -- 172.21.15.37:0/1002755633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 msgr2=0x55ba83239a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83239a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 -- 172.21.15.37:0/1002755633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba832179e0 con 0x55ba83216620 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed737fe640 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed6403dd80 tx=0x7fed64030440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823c9a5fa885cf3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed597fa640 1 -- 172.21.15.37:0/1002755633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64004e00 con 0x55ba83216620 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed597fa640 1 -- 172.21.15.37:0/1002755633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6405e020 con 0x55ba83216620 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.151+0000 7fed597fa640 1 -- 172.21.15.37:0/1002755633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6403db90 con 0x55ba83216620 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 -- 172.21.15.37:0/1002755633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 msgr2=0x55ba83220c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed6403dd80 tx=0x7fed64030440 comp rx=0 tx=0).stop 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 -- 172.21.15.37:0/1002755633 shutdown_connections 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83239a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216f40 0x55ba832394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/1002755633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216620 0x55ba83220c30 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 -- 172.21.15.37:0/1002755633 >> 172.21.15.37:0/1002755633 conn(0x55ba83212b30 msgr2=0x55ba8321e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 -- 172.21.15.37:0/1002755633 shutdown_connections 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.152+0000 7fed7c7bd740 1 -- 172.21.15.37:0/1002755633 wait complete. 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 -- start start 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 Processor -- start 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216620 0x55ba83229570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83226ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fed741aeff0 con 0x55ba83216f40 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed741ae920 con 0x55ba83221800 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fed741abb80 con 0x55ba83216620 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed7c7bd740 10 client.0 _ll_register_callbacks cb 0x7fed640094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58588/0 (socket says 172.21.15.37:58588) 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.153+0000 7fed72ffd640 1 -- 172.21.15.37:0/3007001471 learned_addr learned my addr 172.21.15.37:0/3007001471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed737fe640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216620 0x55ba83229570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed73fff640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83226ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 -- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216620 msgr2=0x55ba83229570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216620 0x55ba83229570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 -- 172.21.15.37:0/3007001471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 msgr2=0x55ba83226ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83226ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 -- 172.21.15.37:0/3007001471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba832271b0 con 0x55ba83216f40 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed73fff640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83226ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed72ffd640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fed68011d10 tx=0x7fed68005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60810c95eebfcf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6801f040 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fed68005240 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6405e020 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed68005440 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed7c7bd740 10 client.17361 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.154+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ba832978d0 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6405e020 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed68025040 con 0x55ba83216f40 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 10 client.17361 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 20 client.17361 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83297e30 0x55ba8329a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 4 client.17361 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba83183200 con 0x55ba83297e30 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.155+0000 7fed737fe640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83297e30 0x55ba8329a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.156+0000 7fed52ffd640 10 client.17361 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.156+0000 7fed737fe640 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83297e30 0x55ba8329a250 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a7268c930cbebb server_cookie=5bf4fe8c7314df09 in_seq=0 out_seq=0 2024-04-25T21:25:36.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:35 smithi098 ceph-mon[32569]: pgmap v538: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-25T21:25:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed52ffd640 1 -- 172.21.15.37:0/3007001471 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba83183200 con 0x55ba83297e30 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed52ffd640 10 client.17361 handle_command_reply: tid=0 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed7c7bd740 1 client.17361 shutdown 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 msgr2=0x55ba83229ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fed68011d10 tx=0x7fed68005e80 comp rx=0 tx=0).stop 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 shutdown_connections 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.173+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x55ba83297e30 0x55ba8329a250 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ba83221800 0x55ba83226ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ba83216f40 0x55ba83229ab0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed52ffd640 0 client.17361 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 --2- 172.21.15.37:0/3007001471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba83216620 0x55ba83229570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 >> 172.21.15.37:0/3007001471 conn(0x55ba83212b30 msgr2=0x55ba83211420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 shutdown_connections 2024-04-25T21:25:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 1 -- 172.21.15.37:0/3007001471 wait complete. 2024-04-25T21:25:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed7c7bd740 20 client.17361 trim_cache size 0 max 16384 2024-04-25T21:25:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-25T21:25:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-25T21:25:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-25T21:25:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-25T21:25:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 msgr2=0x7fed5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 0x7fed5c0a6280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fed6402aa50 tx=0x7fed640058b0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 msgr2=0x7fed7419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.174+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00eb10 tx=0x7fed6c00ef10 comp rx=0 tx=0).stop 2024-04-25T21:25:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 shutdown_connections 2024-04-25T21:25:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed7410c520 0x7fed7419cfd0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fed5c0a3dc0 0x7fed5c0a6280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fed74103b00 0x7fed74198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 --2- 172.21.15.37:0/2206836457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed741031e0 0x7fed74198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 >> 172.21.15.37:0/2206836457 conn(0x7fed74077fd0 msgr2=0x7fed740ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 shutdown_connections 2024-04-25T21:25:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-25T21:25:36.175+0000 7fed79aa9640 1 -- 172.21.15.37:0/2206836457 wait complete. 2024-04-25T21:25:37.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:36 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3557542794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:37.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:36 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3557542794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:37.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:36 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3557542794' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:37.436 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db --cancel --all' 2024-04-25T21:25:37.436 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db --cancel --all 2024-04-25T21:25:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 -- 172.21.15.37:0/475557013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201034b0 msgr2=0x7f2f2010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 --2- 172.21.15.37:0/475557013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201034b0 0x7f2f2010f8e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009990 tx=0x7f2f18009ca0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 -- 172.21.15.37:0/475557013 shutdown_connections 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 --2- 172.21.15.37:0/475557013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201034b0 0x7f2f2010f8e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 --2- 172.21.15.37:0/475557013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f20108a80 0x7f2f20108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 --2- 172.21.15.37:0/475557013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f20102af0 0x7f2f20102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 -- 172.21.15.37:0/475557013 >> 172.21.15.37:0/475557013 conn(0x7f2f20079b20 msgr2=0x7f2f20079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 -- 172.21.15.37:0/475557013 shutdown_connections 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.986+0000 7f2f27108700 1 -- 172.21.15.37:0/475557013 wait complete. 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 Processor -- start 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 -- start start 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 0x7f2f20075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f20108a80 0x7f2f200757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112c10 con 0x7f2f20102af0 2024-04-25T21:25:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112a90 con 0x7f2f201034b0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f27108700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112d90 con 0x7f2f20108a80 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58612/0 (socket says 172.21.15.37:58612) 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 -- 172.21.15.37:0/589199266 learned_addr learned my addr 172.21.15.37:0/589199266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 0x7f2f20075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 -- 172.21.15.37:0/589199266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f20108a80 msgr2=0x7f2f200757c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f20108a80 0x7f2f200757c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 -- 172.21.15.37:0/589199266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 msgr2=0x7f2f20075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 0x7f2f20075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.987+0000 7f2f24ea4700 1 -- 172.21.15.37:0/589199266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a7650 con 0x7f2f20102af0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 0x7f2f20075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f24ea4700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f100060b0 tx=0x7f2f10007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1001a070 con 0x7f2f20102af0 2024-04-25T21:25:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a7940 con 0x7f2f20102af0 2024-04-25T21:25:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f1000de20 con 0x7f2f20102af0 2024-04-25T21:25:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10014720 con 0x7f2f20102af0 2024-04-25T21:25:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.988+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a7ed0 con 0x7f2f20102af0 2024-04-25T21:25:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.989+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f20069730 con 0x7f2f20102af0 2024-04-25T21:25:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.991+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f1001d070 con 0x7f2f20102af0 2024-04-25T21:25:37.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.992+0000 7f2f1dffb700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 0x7f2f0c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:37.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.992+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f2f100d0550 con 0x7f2f20102af0 2024-04-25T21:25:37.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.992+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 0x7f2f0c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:37.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.993+0000 7f2f1ffff700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 0x7f2f0c0a6270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005950 tx=0x7f2f080058e0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:37.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:37.996+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f10010070 con 0x7f2f20102af0 2024-04-25T21:25:38.023 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:38.153 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:37 smithi037 ceph-mon[29195]: pgmap v539: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:38.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:37 smithi098 ceph-mon[32569]: pgmap v539: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:38.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.271+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f2002df80 con 0x7f2f20102af0 2024-04-25T21:25:38.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.272+0000 7f2f1dffb700 1 -- 172.21.15.37:0/589199266 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f2f1009cb90 con 0x7f2f20102af0 2024-04-25T21:25:38.273 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:25:38.273 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:25:38.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 msgr2=0x7f2f0c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:38.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 0x7f2f0c0a6270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005950 tx=0x7f2f080058e0 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 msgr2=0x7f2f20074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f2f100060b0 tx=0x7f2f10007f90 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 shutdown_connections 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f2f0c0a3db0 0x7f2f0c0a6270 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2f20108a80 0x7f2f200757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102af0 0x7f2f20074d40 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 --2- 172.21.15.37:0/589199266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f201034b0 0x7f2f20075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 >> 172.21.15.37:0/589199266 conn(0x7f2f20079b20 msgr2=0x7f2f20101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:38.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.274+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 shutdown_connections 2024-04-25T21:25:38.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:38.275+0000 7f2f27108700 1 -- 172.21.15.37:0/589199266 wait complete. 2024-04-25T21:25:38.276 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:25:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:37 smithi177 ceph-mon[33043]: pgmap v539: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:39.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:38 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/589199266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:39.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:38 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/589199266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:39.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:38 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/589199266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:25:40.135 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:39 smithi037 ceph-mon[29195]: pgmap v540: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:40.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:39 smithi098 ceph-mon[32569]: pgmap v540: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:40.174 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24469 quiesce db / --timeout 60 --expiration 77.7 --await-for 42.0' 2024-04-25T21:25:40.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24469 quiesce db / --timeout 60 --expiration 77.7 --await-for 42.0 2024-04-25T21:25:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:39 smithi177 ceph-mon[33043]: pgmap v540: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-25T21:25:40.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/2367593565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 msgr2=0x7f4a5810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/2367593565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a5810f910 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009990 tx=0x7f4a50009ca0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/2367593565 shutdown_connections 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/2367593565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a5810f910 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/2367593565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58108ad0 0x7f4a58108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/2367593565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58101030 0x7f4a58101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/2367593565 >> 172.21.15.37:0/2367593565 conn(0x7f4a58077fb0 msgr2=0x7f4a580ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.636+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/2367593565 shutdown_connections 2024-04-25T21:25:40.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.637+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/2367593565 wait complete. 2024-04-25T21:25:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.637+0000 7f4a5cfd5700 1 Processor -- start 2024-04-25T21:25:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 -- start start 2024-04-25T21:25:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58101030 0x7f4a58102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58108ad0 0x7f4a58102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112be0 con 0x7f4a581019d0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112a60 con 0x7f4a58108ad0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5cfd5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112d60 con 0x7f4a58101030 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45820/0 (socket says 172.21.15.37:45820) 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4241452893 learned_addr learned my addr 172.21.15.37:0/4241452893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58101030 0x7f4a58102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58108ad0 0x7f4a58102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4241452893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58101030 msgr2=0x7f4a58102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58101030 0x7f4a58102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4241452893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58108ad0 msgr2=0x7f4a58102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.638+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58108ad0 0x7f4a58102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4241452893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581a7520 con 0x7f4a581019d0 2024-04-25T21:25:40.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000d980 tx=0x7f4a4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a4f7fe700 1 -- 172.21.15.37:0/4241452893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40016070 con 0x7f4a581019d0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581a7810 con 0x7f4a581019d0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a4f7fe700 1 -- 172.21.15.37:0/4241452893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a40004830 con 0x7f4a581019d0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a7da0 con 0x7f4a581019d0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.639+0000 7f4a4f7fe700 1 -- 172.21.15.37:0/4241452893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40005540 con 0x7f4a581019d0 2024-04-25T21:25:40.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.641+0000 7f4a4f7fe700 1 -- 172.21.15.37:0/4241452893 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a40004220 con 0x7f4a581019d0 2024-04-25T21:25:40.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.642+0000 7f4a4f7fe700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 0x7f4a440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.642+0000 7f4a4f7fe700 1 -- 172.21.15.37:0/4241452893 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f4a400d0470 con 0x7f4a581019d0 2024-04-25T21:25:40.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.643+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 0x7f4a440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.643+0000 7f4a6785eb80 1 Processor -- start 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.643+0000 7f4a6785eb80 1 -- start start 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 0x7f4a440a6270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007fd0 tx=0x7f4a480079e0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eac318da0 con 0x555eac3c5960 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eac2bf9d0 con 0x555eac3c6280 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eac314d60 con 0x555eac3d1150 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a56d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a56d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.37:52236/0 (socket says 172.21.15.37:52236) 2024-04-25T21:25:40.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.644+0000 7f4a56d9d700 1 -- 172.21.15.37:0/1639529031 learned_addr learned my addr 172.21.15.37:0/1639529031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:40.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 -- 172.21.15.37:0/1639529031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 msgr2=0x555eac3d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 -- 172.21.15.37:0/1639529031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 msgr2=0x555eac3c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 -- 172.21.15.37:0/1639529031 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eac266940 con 0x555eac3d1150 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a50015000 tx=0x7f4a50005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa992594ceb7fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a4cff9700 1 -- 172.21.15.37:0/1639529031 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003a60 con 0x555eac3d1150 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a4cff9700 1 -- 172.21.15.37:0/1639529031 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50026070 con 0x555eac3d1150 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.645+0000 7f4a4cff9700 1 -- 172.21.15.37:0/1639529031 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000bb00 con 0x555eac3d1150 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 -- 172.21.15.37:0/1639529031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 msgr2=0x555eac3d3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4a50015000 tx=0x7f4a50005c90 comp rx=0 tx=0).stop 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 -- 172.21.15.37:0/1639529031 shutdown_connections 2024-04-25T21:25:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d3540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/1639529031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.646+0000 7f4a6785eb80 1 -- 172.21.15.37:0/1639529031 >> 172.21.15.37:0/1639529031 conn(0x555eac2f3700 msgr2=0x555eac3c1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.647+0000 7f4a6785eb80 1 -- 172.21.15.37:0/1639529031 shutdown_connections 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.647+0000 7f4a6785eb80 1 -- 172.21.15.37:0/1639529031 wait complete. 2024-04-25T21:25:40.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.647+0000 7f4a6785eb80 1 -- start start 2024-04-25T21:25:40.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.647+0000 7f4a6785eb80 1 Processor -- start 2024-04-25T21:25:40.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 0x555eac3cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555eac3d3ae0 con 0x555eac3c5960 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x555eac2b7c40 con 0x555eac3d1150 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x555eac3d3c60 con 0x555eac3c6280 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a6785eb80 10 client.0 _ll_register_callbacks cb 0x555eac2b19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45834/0 (socket says 172.21.15.37:45834) 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 0x555eac3cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a5659c700 1 -- 172.21.15.37:0/4090138240 learned_addr learned my addr 172.21.15.37:0/4090138240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4090138240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 msgr2=0x555eac3ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4090138240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 msgr2=0x555eac3d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 -- 172.21.15.37:0/4090138240 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eac2d02e0 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 0x555eac3cdb70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a400cdbc0 tx=0x7f4a4009bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87ebc4c8f01543b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.648+0000 7f4a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a400b5060 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50026070 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f4a4009c870 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a4000ee10 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 10 client.26686 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eac3de3b0 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50026070 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4a400a4bf0 con 0x555eac3c6280 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 10 client.26686 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:40.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 10 client.26686 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 20 client.26686 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.649+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac3e0940 0x555eac3e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.650+0000 7f4a6785eb80 4 client.26686 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.650+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eac326590 con 0x555eac3e0940 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.650+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac3e0940 0x555eac3e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.650+0000 7f4a5659c700 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac3e0940 0x555eac3e2d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e553dba033bbf2f server_cookie=7b0cb82313e92b3a in_seq=0 out_seq=0 2024-04-25T21:25:40.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.650+0000 7f4a3dffb700 10 client.26686 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.651+0000 7f4a3dffb700 1 -- 172.21.15.37:0/4090138240 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555eac326590 con 0x555eac3e0940 2024-04-25T21:25:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.651+0000 7f4a3dffb700 10 client.26686 handle_command_reply: tid=0 2024-04-25T21:25:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.651+0000 7f4a6785eb80 1 client.26686 shutdown 2024-04-25T21:25:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.651+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 msgr2=0x555eac3cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.651+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 0x555eac3cdb70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a400cdbc0 tx=0x7f4a4009bdf0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 shutdown_connections 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac3e0940 0x555eac3e2d30 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/4090138240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c6280 0x555eac3cdb70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 >> 172.21.15.37:0/4090138240 conn(0x555eac2f3700 msgr2=0x555eac3c1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a3dffb700 0 client.26686 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 shutdown_connections 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 1 -- 172.21.15.37:0/4090138240 wait complete. 2024-04-25T21:25:40.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.652+0000 7f4a6785eb80 20 client.26686 trim_cache size 0 max 16384 2024-04-25T21:25:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 Processor -- start 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 -- start start 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 0x555eac3e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3d8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800d540 con 0x555eac3c6280 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800f080 con 0x555eac3d1150 2024-04-25T21:25:40.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800dc10 con 0x555eac3c5960 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a55d9b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45844/0 (socket says 172.21.15.37:45844) 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.673+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2986312610 learned_addr learned my addr 172.21.15.37:0/2986312610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3d8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a5659c700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 0x555eac3e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2986312610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 msgr2=0x555eac3e85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 0x555eac3e85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2986312610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 msgr2=0x555eac3d8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3d8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2986312610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eac3e3310 con 0x555eac3c6280 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a5659c700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 0x555eac3e85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005090 tx=0x7f4a400986b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818ce6c30cd2cdcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a3ffff700 1 -- 172.21.15.37:0/2986312610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40066070 con 0x555eac3c6280 2024-04-25T21:25:40.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a3ffff700 1 -- 172.21.15.37:0/2986312610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a400a07c0 con 0x555eac3c6280 2024-04-25T21:25:40.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.674+0000 7f4a3ffff700 1 -- 172.21.15.37:0/2986312610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4009c670 con 0x555eac3c6280 2024-04-25T21:25:40.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2986312610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 msgr2=0x555eac3e8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005090 tx=0x7f4a400986b0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2986312610 shutdown_connections 2024-04-25T21:25:40.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3d1150 0x555eac3d8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3c5960 0x555eac3e85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2986312610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c6280 0x555eac3e8b10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2986312610 >> 172.21.15.37:0/2986312610 conn(0x555eac2f3700 msgr2=0x555eac3cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2986312610 shutdown_connections 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2986312610 wait complete. 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.675+0000 7f4a6785eb80 1 -- start start 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 Processor -- start 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800e9b0 con 0x555eac3c5960 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800e2e0 con 0x555eac3c6280 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3800b500 con 0x555eac3d1150 2024-04-25T21:25:40.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a6785eb80 10 client.0 _ll_register_callbacks cb 0x555eac2b19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a5659c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45858/0 (socket says 172.21.15.37:45858) 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a5659c700 1 -- 172.21.15.37:0/2605520947 learned_addr learned my addr 172.21.15.37:0/2605520947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a56d9d700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 msgr2=0x555eac3d5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2605520947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 msgr2=0x555eac3d93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a55d9b700 1 -- 172.21.15.37:0/2605520947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eac3d6560 con 0x555eac3c6280 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.676+0000 7f4a5659c700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a55d9b700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d5780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005090 tx=0x7f4a4009bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed9e16919542c06e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-04-25T21:25:40.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mon.2 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40066070 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mon.2 v2:172.21.15.98:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f4a4009c870 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a400a07c0 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mon.2 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a400b5020 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a6785eb80 10 client.26546 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eac447e70 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.677+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mon.2 v2:172.21.15.98:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a400a07c0 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mon.2 v2:172.21.15.98:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4a4009b2d0 con 0x555eac3c6280 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 10 client.26546 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 10 client.26546 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 20 client.26546 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac4482e0 0x555eac44a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 4 client.26546 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x555eac320630 con 0x555eac4482e0 2024-04-25T21:25:40.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a5659c700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac4482e0 0x555eac44a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a3dffb700 10 client.26546 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.678+0000 7f4a5659c700 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac4482e0 0x555eac44a700 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71be4e56431aa56c server_cookie=59a5bca93974a80d in_seq=0 out_seq=0 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a3dffb700 1 -- 172.21.15.37:0/2605520947 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x555eac320630 con 0x555eac4482e0 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a3dffb700 10 client.26546 handle_command_reply: tid=0 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 client.26546 shutdown 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 msgr2=0x555eac3d5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d5780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005090 tx=0x7f4a4009bdf0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 shutdown_connections 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x555eac4482e0 0x555eac44a700 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555eac3d1150 0x555eac3d5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a3dffb700 0 client.26546 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eac3c5960 0x555eac3d93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 --2- 172.21.15.37:0/2605520947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x555eac3c6280 0x555eac3d5780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 >> 172.21.15.37:0/2605520947 conn(0x555eac2f3700 msgr2=0x555eac3cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 shutdown_connections 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 1 -- 172.21.15.37:0/2605520947 wait complete. 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.679+0000 7f4a6785eb80 20 client.26546 trim_cache size 0 max 16384 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 44, 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-04-25T21:25:40.684 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 msgr2=0x7f4a440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 0x7f4a440a6270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007fd0 tx=0x7f4a480079e0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 msgr2=0x7f4a58102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000d980 tx=0x7f4a4000dc90 comp rx=0 tx=0).stop 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 shutdown_connections 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f4a440a3db0 0x7f4a440a6270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a58108ad0 0x7f4a58102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a58101030 0x7f4a58102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 --2- 172.21.15.37:0/4241452893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a581019d0 0x7f4a58102750 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:40.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 >> 172.21.15.37:0/4241452893 conn(0x7f4a58077fb0 msgr2=0x7f4a580ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:40.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 shutdown_connections 2024-04-25T21:25:40.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:40.680+0000 7f4a5cfd5700 1 -- 172.21.15.37:0/4241452893 wait complete. 2024-04-25T21:25:42.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:41 smithi098 ceph-mon[32569]: pgmap v541: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-04-25T21:25:42.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:41 smithi037 ceph-mon[29195]: pgmap v541: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-04-25T21:25:42.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:41 smithi177 ceph-mon[33043]: pgmap v541: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 21 op/s 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 -- 172.21.15.37:0/4013646092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040106840 msgr2=0x7f6040106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 --2- 172.21.15.37:0/4013646092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040106840 0x7f6040106c20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6030009990 tx=0x7f6030009ca0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 -- 172.21.15.37:0/4013646092 shutdown_connections 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 --2- 172.21.15.37:0/4013646092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040102330 0x7f604010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 --2- 172.21.15.37:0/4013646092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040106840 0x7f6040106c20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 --2- 172.21.15.37:0/4013646092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 -- 172.21.15.37:0/4013646092 >> 172.21.15.37:0/4013646092 conn(0x7f6040078980 msgr2=0x7f6040078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 -- 172.21.15.37:0/4013646092 shutdown_connections 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.050+0000 7f6044c87700 1 -- 172.21.15.37:0/4013646092 wait complete. 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 Processor -- start 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 -- start start 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040106840 0x7f60401a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6040111b30 con 0x7f6040102330 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60401119b0 con 0x7f6040101970 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6044c87700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6040111cb0 con 0x7f6040106840 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45876/0 (socket says 172.21.15.37:45876) 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f6037fff700 1 -- 172.21.15.37:0/3076937813 learned_addr learned my addr 172.21.15.37:0/3076937813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.051+0000 7f603ed9d700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040106840 0x7f60401a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f603e59c700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 -- 172.21.15.37:0/3076937813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040106840 msgr2=0x7f60401a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040106840 0x7f60401a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 -- 172.21.15.37:0/3076937813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 msgr2=0x7f6040199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 -- 172.21.15.37:0/3076937813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60401a5470 con 0x7f6040102330 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f603e59c700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6037fff700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f602800d950 tx=0x7f602800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f60377fe700 1 -- 172.21.15.37:0/3076937813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028016070 con 0x7f6040102330 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60401a56a0 con 0x7f6040102330 2024-04-25T21:25:43.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f60377fe700 1 -- 172.21.15.37:0/3076937813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028009df0 con 0x7f6040102330 2024-04-25T21:25:43.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f60377fe700 1 -- 172.21.15.37:0/3076937813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602800f3c0 con 0x7f6040102330 2024-04-25T21:25:43.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.052+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401a5bb0 con 0x7f6040102330 2024-04-25T21:25:43.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.055+0000 7f60377fe700 1 -- 172.21.15.37:0/3076937813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6028019070 con 0x7f6040102330 2024-04-25T21:25:43.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.056+0000 7f60377fe700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 0x7f602c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.056+0000 7f603e59c700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 0x7f602c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.056+0000 7f60377fe700 1 -- 172.21.15.37:0/3076937813 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f60280d0440 con 0x7f6040102330 2024-04-25T21:25:43.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.056+0000 7f603e59c700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 0x7f602c0a6310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f603000b5c0 tx=0x7f6030005a90 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.056+0000 7f604f510b80 1 Processor -- start 2024-04-25T21:25:43.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 -- start start 2024-04-25T21:25:43.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82e3a0 0x56063b83d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b83d680 0x56063b83fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56063b780ec0 con 0x56063b82da80 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56063b727af0 con 0x56063b83d680 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f604f510b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56063b77ce80 con 0x56063b82e3a0 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45888/0 (socket says 172.21.15.37:45888) 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 -- 172.21.15.37:0/1554332449 learned_addr learned my addr 172.21.15.37:0/1554332449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603ed9d700 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b83d680 0x56063b83fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f6037fff700 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82e3a0 0x56063b83d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 -- 172.21.15.37:0/1554332449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82e3a0 msgr2=0x56063b83d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82e3a0 0x56063b83d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 -- 172.21.15.37:0/1554332449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b83d680 msgr2=0x56063b83fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b83d680 0x56063b83fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 -- 172.21.15.37:0/1554332449 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063b6cea60 con 0x56063b82da80 2024-04-25T21:25:43.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.057+0000 7f603e59c700 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6030015000 tx=0x7f603001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ee3d76965210ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f6034ff9700 1 -- 172.21.15.37:0/1554332449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603000be70 con 0x56063b82da80 2024-04-25T21:25:43.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f6034ff9700 1 -- 172.21.15.37:0/1554332449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603000f910 con 0x56063b82da80 2024-04-25T21:25:43.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f6034ff9700 1 -- 172.21.15.37:0/1554332449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300248b0 con 0x56063b82da80 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 -- 172.21.15.37:0/1554332449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 msgr2=0x56063b82de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6030015000 tx=0x7f603001f000 comp rx=0 tx=0).stop 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 -- 172.21.15.37:0/1554332449 shutdown_connections 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b83d680 0x56063b83fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82da80 0x56063b82de60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 --2- 172.21.15.37:0/1554332449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82e3a0 0x56063b83d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 -- 172.21.15.37:0/1554332449 >> 172.21.15.37:0/1554332449 conn(0x56063b75b820 msgr2=0x56063b829f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.058+0000 7f604f510b80 1 -- 172.21.15.37:0/1554332449 shutdown_connections 2024-04-25T21:25:43.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 -- 172.21.15.37:0/1554332449 wait complete. 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 -- start start 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 Processor -- start 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82da80 0x56063b83cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82e3a0 0x56063b83a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56063b83ffb0 con 0x56063b83d680 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x56063b71fd60 con 0x56063b82e3a0 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x56063b840130 con 0x56063b82da80 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f604f510b80 10 client.0 _ll_register_callbacks cb 0x56063b719ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f603ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f603ed9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45890/0 (socket says 172.21.15.37:45890) 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f603ed9d700 1 -- 172.21.15.37:0/2414613766 learned_addr learned my addr 172.21.15.37:0/2414613766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.059+0000 7f603e59c700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82da80 0x56063b83cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6037fff700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82e3a0 0x56063b83a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 -- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82da80 msgr2=0x56063b83cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82da80 0x56063b83cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 -- 172.21.15.37:0/2414613766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82e3a0 msgr2=0x56063b83a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82e3a0 0x56063b83a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 -- 172.21.15.37:0/2414613766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063b738400 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f603ed9d700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f60380042a0 tx=0x7f60380042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbb862461dc45e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603801a070 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f6038016640 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603000f910 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6038026050 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f604f510b80 10 client.17394 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56063b84a810 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.060+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603000f910 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f60380078d0 con 0x56063b83d680 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 10 client.17394 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:43.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 10 client.17394 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 20 client.17394 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b84f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 4 client.17394 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063b78e6b0 con 0x56063b84cda0 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f603e59c700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b84f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f6025ffb700 10 client.17394 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.061+0000 7f603e59c700 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b84f1c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855ae0140cf1561d server_cookie=ca9598dc9c7a6587 in_seq=0 out_seq=0 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f6025ffb700 1 -- 172.21.15.37:0/2414613766 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56063b78e6b0 con 0x56063b84cda0 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f6025ffb700 10 client.17394 handle_command_reply: tid=0 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 client.17394 shutdown 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 msgr2=0x56063b83a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f60380042a0 tx=0x7f60380042d0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 shutdown_connections 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b84f1c0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f6025ffb700 0 client.17394 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b83d680 0x56063b83a5c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b82da80 0x56063b83cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 --2- 172.21.15.37:0/2414613766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82e3a0 0x56063b83a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 >> 172.21.15.37:0/2414613766 conn(0x56063b75b820 msgr2=0x56063b829b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 shutdown_connections 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.062+0000 7f604f510b80 1 -- 172.21.15.37:0/2414613766 wait complete. 2024-04-25T21:25:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.063+0000 7f604f510b80 20 client.17394 trim_cache size 0 max 16384 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 Processor -- start 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 -- start start 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b848e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b854a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60401ab130 con 0x56063b82e3a0 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60401acc70 con 0x56063b82da80 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.082+0000 7f604f510b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60401ab800 con 0x56063b83d680 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45902/0 (socket says 172.21.15.37:45902) 2024-04-25T21:25:43.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 -- 172.21.15.37:0/742503884 learned_addr learned my addr 172.21.15.37:0/742503884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f603e59c700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b848e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f603ed9d700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b854a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 -- 172.21.15.37:0/742503884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 msgr2=0x56063b854a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b854a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 -- 172.21.15.37:0/742503884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 msgr2=0x56063b848e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b848e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 -- 172.21.15.37:0/742503884 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063b82b270 con 0x56063b82e3a0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f603ed9d700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b854a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f603e59c700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b848e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6037fff700 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6028098d50 tx=0x7f602809c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360589caf31d7064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6027fff700 1 -- 172.21.15.37:0/742503884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028014030 con 0x56063b82e3a0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6027fff700 1 -- 172.21.15.37:0/742503884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028010710 con 0x56063b82e3a0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.083+0000 7f6027fff700 1 -- 172.21.15.37:0/742503884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028020040 con 0x56063b82e3a0 2024-04-25T21:25:43.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- 172.21.15.37:0/742503884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 msgr2=0x56063b83c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6028098d50 tx=0x7f602809c4d0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- 172.21.15.37:0/742503884 shutdown_connections 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b854a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b848e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 --2- 172.21.15.37:0/742503884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b83c140 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- 172.21.15.37:0/742503884 >> 172.21.15.37:0/742503884 conn(0x56063b75b820 msgr2=0x56063b835750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- 172.21.15.37:0/742503884 shutdown_connections 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- 172.21.15.37:0/742503884 wait complete. 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 -- start start 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.084+0000 7f604f510b80 1 Processor -- start 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b840340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b8b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60401ac5a0 con 0x56063b82e3a0 2024-04-25T21:25:43.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f60401abed0 con 0x56063b82da80 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f60401a90f0 con 0x56063b83d680 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 10 client.0 _ll_register_callbacks cb 0x56063b719ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45906/0 (socket says 172.21.15.37:45906) 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 -- 172.21.15.37:0/1825108413 learned_addr learned my addr 172.21.15.37:0/1825108413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f603e59c700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b840340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 -- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 msgr2=0x56063b8b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b8b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 -- 172.21.15.37:0/1825108413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 msgr2=0x56063b840340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b840340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 -- 172.21.15.37:0/1825108413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063b8b3800 con 0x56063b82e3a0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6037fff700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f602809c080 tx=0x7f6028010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42477a1abc2625be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6025ffb700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T21:25:43.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60280b9030 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f60280bb2c0 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.085+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6028010710 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6028020030 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 10 client.17400 fetch_fsmap learned FSMap version 27 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56063b8b49c0 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028010710 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f60280d0040 con 0x56063b82e3a0 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 10 client.17400 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 10 client.17400 resolve_mds: validated gid 24469 aka daemon mds.i 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 20 client.17400 populate_metadata read hostname 'smithi037' 2024-04-25T21:25:43.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b8b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 4 client.17400 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.7, "await_for": 42.0}] 2024-04-25T21:25:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 --> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.7, "await_for": 42.0}) -- 0x56063b788750 con 0x56063b84cda0 2024-04-25T21:25:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.086+0000 7f603e59c700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b8b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.087+0000 7f603e59c700 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b8b70c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6985859bd426081 server_cookie=adcb7f3b5ef0f929 in_seq=0 out_seq=0 2024-04-25T21:25:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.087+0000 7f6025ffb700 10 client.17400 ms_handle_connect on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f6025ffb700 1 -- 172.21.15.37:0/1825108413 <== mds.0 v2:172.21.15.177:6838/3063347892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56063b788750 con 0x56063b84cda0 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f6025ffb700 10 client.17400 handle_command_reply: tid=0 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 client.17400 shutdown 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 msgr2=0x56063b840880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f602809c080 tx=0x7f6028010ae0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 shutdown_connections 2024-04-25T21:25:43.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:6838/3063347892,v1:172.21.15.177:6839/3063347892] conn(0x56063b84cda0 0x56063b8b70c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56063b83d680 0x56063b8b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x56063b82da80 0x56063b840340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 --2- 172.21.15.37:0/1825108413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56063b82e3a0 0x56063b840880 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 >> 172.21.15.37:0/1825108413 conn(0x56063b75b820 msgr2=0x56063b83c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f6025ffb700 0 client.17400 ms_handle_reset on v2:172.21.15.177:6838/3063347892 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 shutdown_connections 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 1 -- 172.21.15.37:0/1825108413 wait complete. 2024-04-25T21:25:43.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.100+0000 7f604f510b80 20 client.17400 trim_cache size 0 max 16384 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24469, 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 46, 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "629307da": { 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 46, 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:25:43.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 77.7, 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:43.103 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 msgr2=0x7f602c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 0x7f602c0a6310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f603000b5c0 tx=0x7f6030005a90 comp rx=0 tx=0).stop 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 msgr2=0x7f604019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f602800d950 tx=0x7f602800dc60 comp rx=0 tx=0).stop 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 shutdown_connections 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f602c0a3e50 0x7f602c0a6310 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6040106840 0x7f60401a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6040101970 0x7f6040199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 --2- 172.21.15.37:0/3076937813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040102330 0x7f604019a3f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 >> 172.21.15.37:0/3076937813 conn(0x7f6040078980 msgr2=0x7f60400ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 shutdown_connections 2024-04-25T21:25:43.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:43.101+0000 7f6044c87700 1 -- 172.21.15.37:0/3076937813 wait complete. 2024-04-25T21:25:43.860 INFO:tasks.quiescer:done joining Quiescers 2024-04-25T21:25:43.860 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T21:25:43.870 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:25:43.879 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T21:25:43.891 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T21:25:43.891 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:25:43.891 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:25:43.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T21:25:43.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T21:25:44.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 -- 172.21.15.37:0/3290097794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4102e70 msgr2=0x7f62e4100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 --2- 172.21.15.37:0/3290097794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4102e70 0x7f62e4100450 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009900 tx=0x7f62d402a480 comp rx=0 tx=0).stop 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 -- 172.21.15.37:0/3290097794 shutdown_connections 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 --2- 172.21.15.37:0/3290097794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4100fb0 0x7f62e4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 --2- 172.21.15.37:0/3290097794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4103250 0x7f62e4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 --2- 172.21.15.37:0/3290097794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4102e70 0x7f62e4100450 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 -- 172.21.15.37:0/3290097794 >> 172.21.15.37:0/3290097794 conn(0x7f62e4077390 msgr2=0x7f62e40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.158+0000 7f62e9260640 1 -- 172.21.15.37:0/3290097794 shutdown_connections 2024-04-25T21:25:44.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.159+0000 7f62e9260640 1 -- 172.21.15.37:0/3290097794 wait complete. 2024-04-25T21:25:44.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.159+0000 7f62e9260640 1 Processor -- start 2024-04-25T21:25:44.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.159+0000 7f62e9260640 1 -- start start 2024-04-25T21:25:44.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4102e70 0x7f62e41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4103250 0x7f62e419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62e41143b0 con 0x7f62e4100fb0 2024-04-25T21:25:44.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4114230 con 0x7f62e4103250 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4114530 con 0x7f62e4102e70 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45922/0 (socket says 172.21.15.37:45922) 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 -- 172.21.15.37:0/3577958532 learned_addr learned my addr 172.21.15.37:0/3577958532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e359e640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4103250 0x7f62e419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e259c640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4102e70 0x7f62e41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 -- 172.21.15.37:0/3577958532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4102e70 msgr2=0x7f62e41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4102e70 0x7f62e41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 -- 172.21.15.37:0/3577958532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4103250 msgr2=0x7f62e419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4103250 0x7f62e419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 -- 172.21.15.37:0/3577958532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e419afb0 con 0x7f62e4100fb0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e2d9d640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005940 tx=0x7f62d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d403f030 con 0x7f62e4100fb0 2024-04-25T21:25:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e419b240 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d4003cb0 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.160+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4035070 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.161+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41a73d0 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.161+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4101670 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.163+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62d4042030 con 0x7f62e4100fb0 2024-04-25T21:25:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.163+0000 7f62cbfff640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 0x7f62cc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.163+0000 7f62e259c640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 0x7f62cc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.163+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f62d40f10a0 con 0x7f62e4100fb0 2024-04-25T21:25:44.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.164+0000 7f62e259c640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 0x7f62cc0a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005fd0 tx=0x7f62d800b510 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:43 smithi098 ceph-mon[32569]: pgmap v542: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T21:25:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.168+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62d40bd740 con 0x7f62e4100fb0 2024-04-25T21:25:44.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:43 smithi037 ceph-mon[29195]: pgmap v542: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T21:25:44.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:43 smithi177 ceph-mon[33043]: pgmap v542: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-25T21:25:44.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.380+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f62e4062390 con 0x7f62cc0a3dc0 2024-04-25T21:25:44.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.388+0000 7f62cbfff640 1 -- 172.21.15.37:0/3577958532 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f62e4062390 con 0x7f62cc0a3dc0 2024-04-25T21:25:44.387 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T21:08:29", "first": "2024-04-25T21:09:00", "last": "2024-04-25T21:25:00", "last_pruned": "2024-04-25T21:25:00", "created_count": 17, "pruned_count": 11, "active": true} 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 msgr2=0x7f62cc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 0x7f62cc0a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005fd0 tx=0x7f62d800b510 comp rx=0 tx=0).stop 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 msgr2=0x7f62e41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f62d4005940 tx=0x7f62d40042f0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 shutdown_connections 2024-04-25T21:25:44.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62e4103250 0x7f62e419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f62cc0a3dc0 0x7f62cc0a6280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62e4102e70 0x7f62e41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 --2- 172.21.15.37:0/3577958532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62e4100fb0 0x7f62e41a0630 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 >> 172.21.15.37:0/3577958532 conn(0x7f62e4077390 msgr2=0x7f62e40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 shutdown_connections 2024-04-25T21:25:44.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.390+0000 7f62e9260640 1 -- 172.21.15.37:0/3577958532 wait complete. 2024-04-25T21:25:44.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- 172.21.15.37:0/431461942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8924101b60 msgr2=0x7f892410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 --2- 172.21.15.37:0/431461942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8924101b60 0x7f892410dc70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ca0 tx=0x7f891c02a8b0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- 172.21.15.37:0/431461942 shutdown_connections 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 --2- 172.21.15.37:0/431461942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8924101b60 0x7f892410dc70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 --2- 172.21.15.37:0/431461942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89241011a0 0x7f8924101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 --2- 172.21.15.37:0/431461942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924106fb0 0x7f8924107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- 172.21.15.37:0/431461942 >> 172.21.15.37:0/431461942 conn(0x7f8924076c50 msgr2=0x7f8924077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- 172.21.15.37:0/431461942 shutdown_connections 2024-04-25T21:25:44.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- 172.21.15.37:0/431461942 wait complete. 2024-04-25T21:25:44.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 Processor -- start 2024-04-25T21:25:44.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.505+0000 7f892a39b640 1 -- start start 2024-04-25T21:25:44.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924101b60 0x7f892410ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 0x7f892410f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89241121e0 con 0x7f89241011a0 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8924112060 con 0x7f8924101b60 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f8924112360 con 0x7f8924106fb0 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45940/0 (socket says 172.21.15.37:45940) 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 -- 172.21.15.37:0/2408038331 learned_addr learned my addr 172.21.15.37:0/2408038331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f89237fe640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924101b60 0x7f892410ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8928938640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 0x7f892410f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 -- 172.21.15.37:0/2408038331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 msgr2=0x7f892410f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 0x7f892410f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 -- 172.21.15.37:0/2408038331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924101b60 msgr2=0x7f892410ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924101b60 0x7f892410ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 -- 172.21.15.37:0/2408038331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241a6ae0 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8928938640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 0x7f892410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f8923fff640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f891400c890 tx=0x7f891400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914004220 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241a6dd0 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.506+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89140043c0 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.507+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914004a00 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.507+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241a7290 con 0x7f89241011a0 2024-04-25T21:25:44.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.507+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8924102550 con 0x7f89241011a0 2024-04-25T21:25:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.509+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f89140079b0 con 0x7f89241011a0 2024-04-25T21:25:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.509+0000 7f89217fa640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 0x7f89100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.509+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f89140d0540 con 0x7f89241011a0 2024-04-25T21:25:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.509+0000 7f89237fe640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 0x7f89100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.510+0000 7f89237fe640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 0x7f89100a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f890c005fd0 tx=0x7f890c005850 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.514+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8914014030 con 0x7f89241011a0 2024-04-25T21:25:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.721+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 --> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8924062390 con 0x7f89100a3dc0 2024-04-25T21:25:44.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.729+0000 7f89217fa640 1 -- 172.21.15.37:0/2408038331 <== mgr.24830 v2:172.21.15.37:6800/4256525286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8924062390 con 0x7f89100a3dc0 2024-04-25T21:25:44.729 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 msgr2=0x7f89100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 0x7f89100a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f890c005fd0 tx=0x7f890c005850 comp rx=0 tx=0).stop 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 msgr2=0x7f892410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f891400c890 tx=0x7f891400cd50 comp rx=0 tx=0).stop 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 shutdown_connections 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f89100a3dc0 0x7f89100a6280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8924106fb0 0x7f892410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8924101b60 0x7f892410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 --2- 172.21.15.37:0/2408038331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89241011a0 0x7f892410e700 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 >> 172.21.15.37:0/2408038331 conn(0x7f8924076c50 msgr2=0x7f89240ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:44.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 shutdown_connections 2024-04-25T21:25:44.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.731+0000 7f892a39b640 1 -- 172.21.15.37:0/2408038331 wait complete. 2024-04-25T21:25:44.742 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T21:25:44.753 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T21:25:44.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 -- 172.21.15.37:0/1115819890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103b50 msgr2=0x7fd43810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1115819890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103b50 0x7fd43810c030 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009900 tx=0x7fd42c02a480 comp rx=0 tx=0).stop 2024-04-25T21:25:44.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 -- 172.21.15.37:0/1115819890 shutdown_connections 2024-04-25T21:25:44.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1115819890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd43810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1115819890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103b50 0x7fd43810c030 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1115819890 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103230 0x7fd438103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 -- 172.21.15.37:0/1115819890 >> 172.21.15.37:0/1115819890 conn(0x7fd438077fb0 msgr2=0x7fd4380783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 -- 172.21.15.37:0/1115819890 shutdown_connections 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.857+0000 7fd43e29e640 1 -- 172.21.15.37:0/1115819890 wait complete. 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 Processor -- start 2024-04-25T21:25:44.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 -- start start 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103b50 0x7fd43819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd438198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4381121b0 con 0x7fd438103230 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112030 con 0x7fd438103b50 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43e29e640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112330 con 0x7fd43810c570 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd437fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd437fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45962/0 (socket says 172.21.15.37:45962) 2024-04-25T21:25:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd437fff640 1 -- 172.21.15.37:0/1388453059 learned_addr learned my addr 172.21.15.37:0/1388453059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd4377fe640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103b50 0x7fd43819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.858+0000 7fd43c83b640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd438198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 -- 172.21.15.37:0/1388453059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 msgr2=0x7fd438198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd438198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 -- 172.21.15.37:0/1388453059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103b50 msgr2=0x7fd43819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103b50 0x7fd43819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 -- 172.21.15.37:0/1388453059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438198d10 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd43c83b640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd438198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd437fff640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd42800e9f0 tx=0x7fd42800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42800cd80 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4280108e0 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd438199000 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428018980 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.859+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a52a0 con 0x7fd438103230 2024-04-25T21:25:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.860+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4380691c0 con 0x7fd438103230 2024-04-25T21:25:44.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.861+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd428010430 con 0x7fd438103230 2024-04-25T21:25:44.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.862+0000 7fd4357fa640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 0x7fd4200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:44.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.862+0000 7fd4377fe640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 0x7fd4200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:44.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.862+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fd4280d0b90 con 0x7fd438103230 2024-04-25T21:25:44.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.862+0000 7fd4377fe640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 0x7fd4200a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd42c031440 tx=0x7fd42c005b50 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:44.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:44.866+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd42809d230 con 0x7fd438103230 2024-04-25T21:25:45.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.055+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd438107160 con 0x7fd438103230 2024-04-25T21:25:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.056+0000 7fd4357fa640 1 -- 172.21.15.37:0/1388453059 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+23042 (secure 0 0 0) 0x7fd42809cb30 con 0x7fd438103230 2024-04-25T21:25:45.056 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:25:45.056 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":113,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:25:05.525622+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/1634118111":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/3727603400":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/234140491":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1398754476":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:25:45.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.058+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 msgr2=0x7fd4200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.058+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 0x7fd4200a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd42c031440 tx=0x7fd42c005b50 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.058+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 msgr2=0x7fd43819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.058+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd42800e9f0 tx=0x7fd42800eeb0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 shutdown_connections 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fd4200a3d20 0x7fd4200a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd43810c570 0x7fd438198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd438103b50 0x7fd43819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 --2- 172.21.15.37:0/1388453059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd438103230 0x7fd43819e3e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 >> 172.21.15.37:0/1388453059 conn(0x7fd438077fb0 msgr2=0x7fd4380fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 shutdown_connections 2024-04-25T21:25:45.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.059+0000 7fd43e29e640 1 -- 172.21.15.37:0/1388453059 wait complete. 2024-04-25T21:25:45.070 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T21:00:00.177409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T21:04:56.536477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T21:04:59.835175+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T21:25:45.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T21:25:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 -- 172.21.15.37:0/3042002634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0102190 msgr2=0x7fdda0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 --2- 172.21.15.37:0/3042002634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0102190 0x7fdda0102570 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009900 tx=0x7fdd9002a410 comp rx=0 tx=0).stop 2024-04-25T21:25:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 -- 172.21.15.37:0/3042002634 shutdown_connections 2024-04-25T21:25:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 --2- 172.21.15.37:0/3042002634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 0x7fdda010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 --2- 172.21.15.37:0/3042002634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102b40 0x7fdda0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 --2- 172.21.15.37:0/3042002634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0102190 0x7fdda0102570 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 -- 172.21.15.37:0/3042002634 >> 172.21.15.37:0/3042002634 conn(0x7fdda00fd990 msgr2=0x7fdda00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 -- 172.21.15.37:0/3042002634 shutdown_connections 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 -- 172.21.15.37:0/3042002634 wait complete. 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.176+0000 7fdda6410640 1 Processor -- start 2024-04-25T21:25:45.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 -- start start 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102190 0x7fdda019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 0x7fdda019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdda01121c0 con 0x7fdda0198610 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0112040 con 0x7fdda0107260 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda6410640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fdda0112340 con 0x7fdda0102190 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45970/0 (socket says 172.21.15.37:45970) 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 -- 172.21.15.37:0/627118031 learned_addr learned my addr 172.21.15.37:0/627118031 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdd9ffff640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102190 0x7fdda019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdd9f7fe640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 0x7fdda019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 -- 172.21.15.37:0/627118031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102190 msgr2=0x7fdda019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102190 0x7fdda019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 -- 172.21.15.37:0/627118031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 msgr2=0x7fdda019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 0x7fdda019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.177+0000 7fdda49ad640 1 -- 172.21.15.37:0/627118031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda0199130 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdda49ad640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800eb10 tx=0x7fdd9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98004b20 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd98004cc0 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda0199420 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98010670 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.178+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda01a55d0 con 0x7fdda0198610 2024-04-25T21:25:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.179+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda00691c0 con 0x7fdda0198610 2024-04-25T21:25:45.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.180+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd98005020 con 0x7fdda0198610 2024-04-25T21:25:45.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.181+0000 7fdd9d7fa640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.181+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7fdd980d0d00 con 0x7fdda0198610 2024-04-25T21:25:45.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.181+0000 7fdd9ffff640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.181+0000 7fdd9ffff640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd90031440 tx=0x7fdd90005b50 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.185+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd9809d3a0 con 0x7fdda0198610 2024-04-25T21:25:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.371+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdda0102b40 con 0x7fdda0198610 2024-04-25T21:25:45.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.371+0000 7fdd9d7fa640 1 -- 172.21.15.37:0/627118031 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fdd9809cca0 con 0x7fdda0198610 2024-04-25T21:25:45.371 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T21:25:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 msgr2=0x7fdd8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd90031440 tx=0x7fdd90005b50 comp rx=0 tx=0).stop 2024-04-25T21:25:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 msgr2=0x7fdda0198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800eb10 tx=0x7fdd9800c2d0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 shutdown_connections 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdda0198610 0x7fdda0198ac0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdda0107260 0x7fdda019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fdd8c0a3d20 0x7fdd8c0a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 --2- 172.21.15.37:0/627118031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdda0102190 0x7fdda019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 >> 172.21.15.37:0/627118031 conn(0x7fdda00fd990 msgr2=0x7fdda00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.373+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 shutdown_connections 2024-04-25T21:25:45.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.374+0000 7fdda6410640 1 -- 172.21.15.37:0/627118031 wait complete. 2024-04-25T21:25:45.385 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T21:25:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 -- 172.21.15.37:0/1975283134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 msgr2=0x7f761010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 --2- 172.21.15.37:0/1975283134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f761010dd30 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7608009ca0 tx=0x7f760802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 -- 172.21.15.37:0/1975283134 shutdown_connections 2024-04-25T21:25:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 --2- 172.21.15.37:0/1975283134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f761010dd30 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 --2- 172.21.15.37:0/1975283134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7610101a40 0x7f7610105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 --2- 172.21.15.37:0/1975283134 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 0x7f7610101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 -- 172.21.15.37:0/1975283134 >> 172.21.15.37:0/1975283134 conn(0x7f76100771d0 msgr2=0x7f76100775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 -- 172.21.15.37:0/1975283134 shutdown_connections 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.489+0000 7f7616e7b640 1 -- 172.21.15.37:0/1975283134 wait complete. 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 Processor -- start 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 -- start start 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 0x7f7610199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f761019a000 0x7f76101a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76101110b0 con 0x7f7610106160 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7610110f30 con 0x7f7610101090 2024-04-25T21:25:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f7616e7b640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f7610111230 con 0x7f761019a000 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f760ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f760ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45990/0 (socket says 172.21.15.37:45990) 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.490+0000 7f760ffff640 1 -- 172.21.15.37:0/3886365492 learned_addr learned my addr 172.21.15.37:0/3886365492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f7614c17640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 0x7f7610199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 -- 172.21.15.37:0/3886365492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f761019a000 msgr2=0x7f76101a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f761019a000 0x7f76101a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 -- 172.21.15.37:0/3886365492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 msgr2=0x7f7610199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 0x7f7610199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 -- 172.21.15.37:0/3886365492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76101a4d80 con 0x7f7610106160 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760ffff640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7604005fd0 tx=0x7f760400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604017070 con 0x7f7610106160 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76101a5070 con 0x7f7610106160 2024-04-25T21:25:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7604009c30 con 0x7f7610106160 2024-04-25T21:25:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604003d30 con 0x7f7610106160 2024-04-25T21:25:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.491+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76101a55c0 con 0x7f7610106160 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.492+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7610102560 con 0x7f7610106160 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.493+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7604003ed0 con 0x7f7610106160 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.494+0000 7f760dffb640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.494+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f76040d04b0 con 0x7f7610106160 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.494+0000 7f7614c17640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.494+0000 7f7614c17640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 0x7f75f80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7600005fd0 tx=0x7f7600005850 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.498+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7604020030 con 0x7f7610106160 2024-04-25T21:25:45.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.690+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7610101a40 con 0x7f7610106160 2024-04-25T21:25:45.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.690+0000 7f760dffb640 1 -- 172.21.15.37:0/3886365492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f760409cb50 con 0x7f7610106160 2024-04-25T21:25:45.690 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T21:25:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.692+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 msgr2=0x7f75f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.692+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 0x7f75f80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7600005fd0 tx=0x7f7600005850 comp rx=0 tx=0).stop 2024-04-25T21:25:45.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.692+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 msgr2=0x7f7610199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.692+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7604005fd0 tx=0x7f760400dde0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 shutdown_connections 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f75f80a3d20 0x7f75f80a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f761019a000 0x7f76101a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7610106160 0x7f7610199a70 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 --2- 172.21.15.37:0/3886365492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7610101090 0x7f7610199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 >> 172.21.15.37:0/3886365492 conn(0x7f76100771d0 msgr2=0x7f76100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 shutdown_connections 2024-04-25T21:25:45.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.693+0000 7f7616e7b640 1 -- 172.21.15.37:0/3886365492 wait complete. 2024-04-25T21:25:45.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T21:25:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 -- 172.21.15.37:0/3341653631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780806a840 msgr2=0x7f780806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 --2- 172.21.15.37:0/3341653631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780806a840 0x7f780806ac20 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802a410 comp rx=0 tx=0).stop 2024-04-25T21:25:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 -- 172.21.15.37:0/3341653631 shutdown_connections 2024-04-25T21:25:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 --2- 172.21.15.37:0/3341653631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 0x7f780810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 --2- 172.21.15.37:0/3341653631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806b1f0 0x7f7808106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 --2- 172.21.15.37:0/3341653631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780806a840 0x7f780806ac20 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 -- 172.21.15.37:0/3341653631 >> 172.21.15.37:0/3341653631 conn(0x7f7808077eb0 msgr2=0x7f78080782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 -- 172.21.15.37:0/3341653631 shutdown_connections 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.809+0000 7f780e08a640 1 -- 172.21.15.37:0/3341653631 wait complete. 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 Processor -- start 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 -- start start 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 0x7f7808198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 0x7f7808198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7808112230 con 0x7f780819cfd0 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78081120b0 con 0x7f780806a840 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f780e08a640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f78081123b0 con 0x7f7808107270 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f7807fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f7807fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46002/0 (socket says 172.21.15.37:46002) 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f7807fff640 1 -- 172.21.15.37:0/3159195615 learned_addr learned my addr 172.21.15.37:0/3159195615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f7806ffd640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 0x7f7808198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.810+0000 7f78077fe640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 0x7f7808198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 -- 172.21.15.37:0/3159195615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 msgr2=0x7f7808198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 0x7f7808198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 -- 172.21.15.37:0/3159195615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 msgr2=0x7f7808198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 0x7f7808198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 -- 172.21.15.37:0/3159195615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819dac0 con 0x7f780819cfd0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f78077fe640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 0x7f7808198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7807fff640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f780000eaa0 tx=0x7f780000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780000cdb0 con 0x7f780819cfd0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7800005020 con 0x7f780819cfd0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780819ddb0 con 0x7f780819cfd0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000116d0 con 0x7f780819cfd0 2024-04-25T21:25:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.811+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a55a0 con 0x7f780819cfd0 2024-04-25T21:25:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.812+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78080691c0 con 0x7f780819cfd0 2024-04-25T21:25:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.813+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78000051d0 con 0x7f780819cfd0 2024-04-25T21:25:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.814+0000 7f7804ff9640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 0x7f77f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.814+0000 7f78077fe640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 0x7f77f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:45.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.814+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f78000d0940 con 0x7f780819cfd0 2024-04-25T21:25:45.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.814+0000 7f78077fe640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 0x7f77f00a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f8005a20 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:45.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:45.818+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7800016030 con 0x7f780819cfd0 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: from='client.17406 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: pgmap v543: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: from='client.17412 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1388453059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/627118031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:25:45.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3886365492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:25:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.006+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f780806b1f0 con 0x7f780819cfd0 2024-04-25T21:25:46.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.006+0000 7f7804ff9640 1 -- 172.21.15.37:0/3159195615 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f7800085e20 con 0x7f780819cfd0 2024-04-25T21:25:46.006 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 msgr2=0x7f77f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 0x7f77f00a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f77f8000c00 tx=0x7f77f8005a20 comp rx=0 tx=0).stop 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 msgr2=0x7f780819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f780000eaa0 tx=0x7f780000ef60 comp rx=0 tx=0).stop 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 shutdown_connections 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f780819cfd0 0x7f780819d480 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f77f00a3d20 0x7f77f00a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7808107270 0x7f7808198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 --2- 172.21.15.37:0/3159195615 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f780806a840 0x7f7808198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 >> 172.21.15.37:0/3159195615 conn(0x7f7808077eb0 msgr2=0x7f7808108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 shutdown_connections 2024-04-25T21:25:46.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.008+0000 7f780e08a640 1 -- 172.21.15.37:0/3159195615 wait complete. 2024-04-25T21:25:46.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T21:25:46.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- 172.21.15.37:0/118911107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c102190 msgr2=0x7f9c2c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 --2- 172.21.15.37:0/118911107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c102190 0x7f9c2c102570 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009900 tx=0x7f9c2802a410 comp rx=0 tx=0).stop 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- 172.21.15.37:0/118911107 shutdown_connections 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 --2- 172.21.15.37:0/118911107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 --2- 172.21.15.37:0/118911107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102b40 0x7f9c2c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 --2- 172.21.15.37:0/118911107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c102190 0x7f9c2c102570 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- 172.21.15.37:0/118911107 >> 172.21.15.37:0/118911107 conn(0x7f9c2c079310 msgr2=0x7f9c2c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- 172.21.15.37:0/118911107 shutdown_connections 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- 172.21.15.37:0/118911107 wait complete. 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 Processor -- start 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.122+0000 7f9c33831640 1 -- start start 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 0x7f9c2c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c1121a0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c112020 con 0x7f9c2c107260 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2c112320 con 0x7f9c2c102190 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46022/0 (socket says 172.21.15.37:46022) 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 -- 172.21.15.37:0/2532114562 learned_addr learned my addr 172.21.15.37:0/2532114562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c315cd640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 0x7f9c2c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c30dcc640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 -- 172.21.15.37:0/2532114562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 msgr2=0x7f9c2c19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 0x7f9c2c19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 -- 172.21.15.37:0/2532114562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 msgr2=0x7f9c2c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 -- 172.21.15.37:0/2532114562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2c199100 con 0x7f9c2c1985e0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c315cd640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 0x7f9c2c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c30dcc640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c31dce640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009fd0 tx=0x7f9c2400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24017070 con 0x7f9c2c1985e0 2024-04-25T21:25:46.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2c1993f0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c24003e20 con 0x7f9c2c1985e0 2024-04-25T21:25:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.123+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24010460 con 0x7f9c2c1985e0 2024-04-25T21:25:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.124+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c2c1a55a0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.124+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c2c0691c0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.126+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c24003970 con 0x7f9c2c1985e0 2024-04-25T21:25:46.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.126+0000 7f9c227fc640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 0x7f9c180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:25:46.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.126+0000 7f9c315cd640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 0x7f9c180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:25:46.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.126+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8513+0+0 (secure 0 0 0) 0x7f9c240d05a0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.127+0000 7f9c315cd640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 0x7f9c180a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c28000c00 tx=0x7f9c28031400 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:25:46.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.130+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c2409cc40 con 0x7f9c2c1985e0 2024-04-25T21:25:46.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: from='client.17406 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: pgmap v543: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-25T21:25:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: from='client.17412 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1388453059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:25:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/627118031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:25:46.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3886365492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: from='client.17406 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: pgmap v543: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: from='client.17412 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1388453059' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/627118031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:25:46.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3886365492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:25:46.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.367+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9c2c199ac0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.375+0000 7f9c227fc640 1 -- 172.21.15.37:0/2532114562 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f9c2c199ac0 con 0x7f9c2c1985e0 2024-04-25T21:25:46.385 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:25:46.385 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 24830, 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4256525286 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4256525286 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/4256525286", 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-04-25T21:08:28.094451+0000", 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24824, 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-04-25T21:25:46.387 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.390 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.391 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.392 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T21:25:46.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T21:25:46.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T21:25:46.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:25:46.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.407 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T21:25:46.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.413 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T21:25:46.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T21:25:46.421 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:25:46.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:25:46.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:25:46.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.433 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T21:25:46.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T21:25:46.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:25:46.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:25:46.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:25:46.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.458 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.462 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.463 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:25:46.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T21:25:46.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T21:25:46.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T21:25:46.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:25:46.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T21:25:46.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:25:46.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:25:46.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.480 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:25:46.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.483 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.486 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T21:25:46.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T21:25:46.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.497 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.500 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:25:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:25:46.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:25:46.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:25:46.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:25:46.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:25:46.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:25:46.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:25:46.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:25:46.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:25:46.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T21:25:46.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.536 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T21:25:46.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:46.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.542 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:46.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T21:25:46.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:46.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T21:25:46.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.557 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T21:25:46.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:25:46.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:25:46.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.577 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.578 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.580 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T21:25:46.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.588 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.589 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.600 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.602 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.605 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T21:25:46.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.611 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.613 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.614 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T21:25:46.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T21:25:46.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T21:25:46.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.622 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.627 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.630 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.635 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:46.637 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:25:46.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.639 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.648 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T21:25:46.650 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.651 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.655 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.658 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.659 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:25:46.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:46.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.668 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.672 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.674 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.675 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T21:25:46.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:46.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T21:25:46.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:46.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.701 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:46.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T21:25:46.705 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.706 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.713 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.714 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.718 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.720 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.727 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.728 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T21:25:46.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T21:25:46.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.733 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.734 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.735 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.736 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.741 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.749 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.751 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:25:46.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.764 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:46.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T21:25:46.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.777 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.780 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.782 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.783 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.785 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:25:46.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.794 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T21:25:46.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.801 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24853, 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.806 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.812 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T21:25:46.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T21:25:46.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.815 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:25:46.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:46.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.820 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.821 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T21:25:46.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.826 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T21:25:46.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.834 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T21:25:46.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.838 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T21:25:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:25:46.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:46.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.858 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.859 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:25:46.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T21:25:46.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T21:25:46.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T21:25:46.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.870 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:25:46.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T21:25:46.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:25:46.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.887 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T21:25:46.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.893 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.894 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T21:25:46.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.916 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:25:46.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:25:46.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:25:46.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:25:46.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.933 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:25:46.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:25:46.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T21:25:46.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T21:25:46.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.953 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.954 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T21:25:46.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.965 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.967 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.968 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T21:25:46.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.976 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.977 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.981 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.982 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:46.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:25:46.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.988 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:46.991 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T21:25:46.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.997 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.998 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:46.999 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T21:25:47.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.012 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.016 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.017 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:25:47.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.027 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.033 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.035 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.040 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.042 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.043 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.046 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.048 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.049 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.050 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.052 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.053 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.054 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.055 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.057 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.059 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.060 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T21:25:47.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.069 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.073 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.080 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.081 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:25:47.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.089 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:47.090 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.094 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.099 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.102 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.111 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.113 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.114 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.116 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.119 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.120 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.126 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T21:25:47.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T21:25:47.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T21:25:47.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T21:25:47.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.142 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.144 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.145 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.149 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.151 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.153 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T21:25:47.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.160 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T21:25:47.162 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:25:47.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:25:47.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:25:47.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:25:47.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.176 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.181 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.182 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.192 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T21:25:47.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.198 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.202 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T21:25:47.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T21:25:47.207 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T21:25:47.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.209 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.210 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.212 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.215 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-25T21:25:47.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-25T21:25:47.224 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:47.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:47.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:25:47.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.233 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.236 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-25T21:25:47.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.240 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:25:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:25:47.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-25T21:25:47.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-25T21:25:47.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-25T21:25:47.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-25T21:25:47.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.250 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-25T21:25:47.252 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:25:47.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.258 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:25:47.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:25:47.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-25T21:25:47.263 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-25T21:25:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-25T21:25:47.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-25T21:25:47.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.272 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.284 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.298 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.299 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.300 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-25T21:25:47.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-25T21:25:47.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.312 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-25T21:25:47.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:25:47.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:25:47.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:25:47.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.320 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.322 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.333 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:25:47.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:25:47.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:25:47.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:25:47.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:25:47.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.357 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.360 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.368 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.371 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:47.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:47.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.379 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.380 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.381 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.383 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:25:47.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.394 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.395 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.399 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:25:47.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:25:47.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.404 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.406 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-25T21:25:47.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-25T21:25:47.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-25T21:25:47.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:25:47.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.414 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:25:47.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.426 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.428 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-25T21:25:47.430 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.434 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.441 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.445 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.446 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.450 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.451 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.455 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.457 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-25T21:25:47.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-25T21:25:47.463 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.464 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.466 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.469 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.472 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.473 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.474 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-25T21:25:47.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.480 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:47.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:25:47.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.491 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-25T21:25:47.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-25T21:25:47.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-25T21:25:47.501 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-25T21:25:47.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.506 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.514 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.515 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.516 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.521 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.526 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.527 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.528 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.533 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.534 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.537 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-25T21:25:47.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.545 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.546 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.549 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.551 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.554 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.556 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.558 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.559 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.566 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.569 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.570 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:25:47.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:25:47.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.577 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-25T21:25:47.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.586 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.594 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.598 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:25:47.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.606 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-25T21:25:47.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-25T21:25:47.610 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-25T21:25:47.611 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-25T21:25:47.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T21:25:47.615 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T21:25:47.616 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.617 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-25T21:25:47.618 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 91, 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.619 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2720790038 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4129856452 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.620 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1886322006 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.621 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 319143536 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1331251456 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:47.622 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4015007766 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T21:25:47.623 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:47.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:46 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3159195615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:25:47.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:46 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2532114562' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:25:47.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:46 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3159195615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:25:47.625 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:46 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2532114562' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:25:47.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 msgr2=0x7f9c180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:47.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 0x7f9c180a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c28000c00 tx=0x7f9c28031400 comp rx=0 tx=0).stop 2024-04-25T21:25:47.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 msgr2=0x7f9c2c198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:25:47.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009fd0 tx=0x7f9c2400b260 comp rx=0 tx=0).stop 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 shutdown_connections 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c2c1985e0 0x7f9c2c198a90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9c2c107260 0x7f9c2c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f9c180a3d20 0x7f9c180a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 --2- 172.21.15.37:0/2532114562 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c2c102190 0x7f9c2c19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 >> 172.21.15.37:0/2532114562 conn(0x7f9c2c079310 msgr2=0x7f9c2c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 shutdown_connections 2024-04-25T21:25:47.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:25:46.381+0000 7f9c33831640 1 -- 172.21.15.37:0/2532114562 wait complete. 2024-04-25T21:25:47.631 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T21:25:47.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T21:25:47.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:46 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3159195615' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:25:47.632 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:46 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2532114562' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:25:47.893 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:47.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:47 smithi037 ceph-mon[29195]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:48.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:47 smithi098 ceph-mon[32569]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:47 smithi177 ceph-mon[33043]: pgmap v544: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:49.706 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2467, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2467, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 741633, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 783683, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.262074284, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.084903976, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.069607042, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.082827088, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2466, 2024-04-25T21:25:49.707 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.093877936, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000038068 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1507, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 1494, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1949398, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 571942, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-25T21:25:49.708 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.190381181, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.059311868, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.069019126, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.056942123, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1493, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.062659146, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000041968 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1671504, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 213488 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 260, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 748, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 704238, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 8, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 4, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.019418271, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013395035, 2024-04-25T21:25:49.710 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002911539, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000018066, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 257, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.016556640, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000064422 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 703920 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T21:25:49.711 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 5665, 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.187369583, 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000033074 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 0, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 0, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 0, 2024-04-25T21:25:49.714 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 1, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:25:49.715 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:25:49.716 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:25:49.717 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 0 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 0, 2024-04-25T21:25:49.718 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-04-25T21:25:49.719 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 0, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 9, 2024-04-25T21:25:49.720 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 2, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 2, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 4194304, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 4196796, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 4196655, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 9 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 3, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 3, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 0, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 1, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 1, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 0, 2024-04-25T21:25:49.722 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 50972, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 215416 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 0, 2024-04-25T21:25:49.723 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 0, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 0, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-04-25T21:25:49.724 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.725 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-04-25T21:25:49.726 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-04-25T21:25:49.727 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.728 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.729 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.730 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-04-25T21:25:49.732 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.733 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.734 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 1, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 4, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 3, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 3, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 0, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 0, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 4797, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:25:49.736 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:25:49.737 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:25:49.738 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 190962, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 55, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:25:49.739 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8032, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 167, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 8512, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 13, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T21:25:49.740 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 3591, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 21, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 3, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 3591, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3591, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 2.264969843, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000630735 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-04-25T21:25:49.741 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3591, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 3592 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 3591, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 3591, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 2352, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 1237, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-04-25T21:25:49.742 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 1, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-04-25T21:25:49.743 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-04-25T21:25:49.744 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 113, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 25, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 5, 2024-04-25T21:25:49.745 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 7, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 2, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 0, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 2, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 0 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 0, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-04-25T21:25:49.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 3974, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 2311558, 2024-04-25T21:25:49.747 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 3974, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3974, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 2311558, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:49.748 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 3591, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 2497, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3591, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 2497, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-04-25T21:25:49.749 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 3591, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 3591, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3591, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 3591, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.750 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:49.751 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x5589fc59a6e0": { 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:49.752 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:49.753 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:49.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:49 smithi037 ceph-mon[29195]: pgmap v545: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:50.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:49 smithi098 ceph-mon[32569]: pgmap v545: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:50.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:49 smithi177 ceph-mon[33043]: pgmap v545: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2024-04-25T21:25:50.522 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:25:50.523 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:25:50.523 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T21:25:50.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T21:25:50.774 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:51.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:50 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:25:51.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:50 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:25:51.240 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:50 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:25:52.127 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:51 smithi037 ceph-mon[29195]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2024-04-25T21:25:52.127 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:51 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:25:52.127 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:51 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:25:52.127 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:51 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:25:52.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:51 smithi098 ceph-mon[32569]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2024-04-25T21:25:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:51 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:25:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:51 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:25:52.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:51 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:25:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:51 smithi177 ceph-mon[33043]: pgmap v546: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 19 op/s 2024-04-25T21:25:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:51 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:25:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:51 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:25:52.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:51 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' 2024-04-25T21:25:52.527 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:52.527 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:25:52.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-25T21:25:52.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-25T21:25:52.527 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001348568, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000295641, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:52.528 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 359, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 346, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1631296, 2024-04-25T21:25:52.529 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 212704, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.065113523, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.024174181, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.040880569, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.007359852, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 346, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017380112, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000050231 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.530 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1631296, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 212704 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 3, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 263, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 480, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 280864, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T21:25:52.531 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.022284834, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.016466642, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002137873, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 260, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.024456964, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000094065 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 280864 2024-04-25T21:25:52.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:25:52.533 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:25:52.534 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:25:52.535 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T21:25:52.536 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 358, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1578126, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 358, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 358, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1578126, 2024-04-25T21:25:52.537 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:52.538 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:53.226 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T21:25:53.226 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T21:25:53.226 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T21:25:53.227 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T21:25:53.227 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T21:25:53.227 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:25:53.227 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:25:53.227 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T21:25:53.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T21:25:53.483 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:54.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:53 smithi098 ceph-mon[32569]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T21:25:54.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:53 smithi037 ceph-mon[29195]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T21:25:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:53 smithi177 ceph-mon[33043]: pgmap v547: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-04-25T21:25:55.293 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:55.293 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 1235, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 1237, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 414905, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 392731, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 7, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 2, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.147201972, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.062735176, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.030765083, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.038162235, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 1235, 2024-04-25T21:25:55.294 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.068257264, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000055269 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2671, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2656, 2024-04-25T21:25:55.295 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2258517, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 941903, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.218705815, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.066963742, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.083239146, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.065477895, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 2656, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.085025180, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000032012 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.296 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1675088, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 215216 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 263, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1828, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 706677, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 9, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 5, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.019553169, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.013188926, 2024-04-25T21:25:55.297 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.003182168, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000063295, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 260, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.021095287, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000081135 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.298 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 705712 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 1, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-MDSRank": { 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 5976, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.151151209, 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000025293 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "finisher-PurgeQueue": { 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "queue_len": 0, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "complete_latency": { 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": { 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "request": 0, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "reply": 0, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "reply_latency": { 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.300 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_reply": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "forward": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_complete": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_fetch_keys": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_commit": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_split": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_merge": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_top": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_bottom": 0, 2024-04-25T21:25:55.301 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pin_tail": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_pinned": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_expired": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "inodes_with_caps": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "subtrees": 1, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_hit": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_forward": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_discover": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_remote_ino": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "traverse_lock": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "load_cent": 0, 2024-04-25T21:25:55.302 INFO:teuthology.orchestra.run.smithi037.stdout: "q": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "exported": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "exported_inodes": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "imported": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "imported_inodes": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_dir_fetch": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "openino_peer_discover": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rfiles": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rbytes": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "root_rsnaps": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_set_tag": 0, 2024-04-25T21:25:55.303 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_file_inodes": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_cap_release": 0, 2024-04-25T21:25:55.304 INFO:teuthology.orchestra.run.smithi037.stdout: "process_request_cap_release": 0 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_cache": { 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_update_receipt": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_try_discover": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_send_discover": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "dir_handle_discover": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_delayed": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_created": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_enqueued": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_reintegrated": 0, 2024-04-25T21:25:55.305 INFO:teuthology.orchestra.run.smithi037.stdout: "strays_migrated": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_processing": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_started": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "recovery_completed": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_exportdir": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_flush": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_fragstats": 0, 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ireq_inodestats": 0 2024-04-25T21:25:55.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_log": { 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evlrg": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evadd": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evex": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evtrm": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "ev": 5, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evexg": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "evexd": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "segadd": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "segex": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "segtrm": 0, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "seg": 2, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "segmjr": 2, 2024-04-25T21:25:55.307 INFO:teuthology.orchestra.run.smithi037.stdout: "segexg": 0, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "segexd": 0, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "expos": 4194304, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "wrpos": 4195815, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "rdpos": 4195674, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "jlat": { 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "replayed": 5 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_mem": { 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 3, 2024-04-25T21:25:55.308 INFO:teuthology.orchestra.run.smithi037.stdout: "ino+": 3, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "ino-": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dir": 1, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dir+": 1, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dir-": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dn": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dn+": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "dn-": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "cap": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "cap+": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "cap-": 0, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "rss": 50672, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "heap": 215416 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.309 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_server": { 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_client_request": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "dispatch_server_request": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_request": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_client_session": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "handle_peer_request": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "req_create_latency": { 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getattr_latency": { 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.310 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getfilelock_latency": { 2024-04-25T21:25:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.311 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "req_link_latency": { 2024-04-25T21:25:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.313 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookup_latency": { 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookuphash_latency": { 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.314 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupino_latency": { 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupname_latency": { 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupparent_latency": { 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.315 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lookupsnap_latency": { 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "req_lssnap_latency": { 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mkdir_latency": { 2024-04-25T21:25:55.316 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mknod_latency": { 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.317 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "req_mksnap_latency": { 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "req_open_latency": { 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "req_readdir_latency": { 2024-04-25T21:25:55.318 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rename_latency": { 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "req_renamesnap_latency": { 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.319 INFO:teuthology.orchestra.run.smithi037.stdout: "req_snapdiff_latency": { 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmdir_latency": { 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmsnap_latency": { 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "req_rmxattr_latency": { 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setattr_latency": { 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setdirlayout_latency": { 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.321 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setfilelock_latency": { 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setlayout_latency": { 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "req_setxattr_latency": { 2024-04-25T21:25:55.322 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "req_symlink_latency": { 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "req_unlink_latency": { 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.323 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "req_getvxattr_latency": { 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_sessions": { 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "session_count": 1, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "session_add": 2, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "session_remove": 1, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_open": 2, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "sessions_stale": 0, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "total_load": 0, 2024-04-25T21:25:55.324 INFO:teuthology.orchestra.run.smithi037.stdout: "average_load": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "avg_session_uptime": 3552, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "mdthresh_evicted": 0 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:25:55.325 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:25:55.326 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 199025, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 58, 2024-04-25T21:25:55.327 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 8032, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 167, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 8512, 2024-04-25T21:25:55.328 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 13, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "objecter": { 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_active": 0, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_laggy": 0, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send": 3523, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_send_bytes": 21, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_resend": 3, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_reply": 3523, 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "op_latency": { 2024-04-25T21:25:55.329 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3523, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 1.889974828, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000536467 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op_inflight": 0, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "oplen_avg": { 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3523, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 3524 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op": 3523, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op_r": 3523, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op_w": 0, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op_rmw": 0, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "op_pg": 0, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_stat": 2280, 2024-04-25T21:25:55.330 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_create": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_read": 1241, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_write": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writefull": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_writesame": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_append": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_zero": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_truncate": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_delete": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_mapext": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_sparse_read": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_clonerange": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_getxattr": 1, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_setxattr": 0, 2024-04-25T21:25:55.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_rmxattr": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_call": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_watch": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_notify": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "osdop_other": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_active": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_send": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_resend": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "linger_ping": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_active": 0, 2024-04-25T21:25:55.332 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_send": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "poolop_resend": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_active": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_send": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "poolstat_resend": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_active": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_send": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "statfs_resend": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "command_active": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "command_send": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "command_resend": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "map_epoch": 113, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "map_full": 0, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "map_inc": 25, 2024-04-25T21:25:55.333 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_sessions": 6, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_open": 8, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_session_close": 2, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_laggy": 0, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_wr": 0, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_rd": 2, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_del": 0 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "oft": { 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_objs": 0, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_updates": 0, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "omap_total_removes": 0 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.334 INFO:teuthology.orchestra.run.smithi037.stdout: "osd-slow-ops": { 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "slow_ops_count": 0 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "purge_queue": { 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executing_high_water": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed_ops": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_executed": 0, 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: "pq_item_in_journal": 0 2024-04-25T21:25:55.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 3908, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 2300865, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 3908, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3908, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 2300865, 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:55.336 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_bytes": { 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 3523, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 1516, 2024-04-25T21:25:55.337 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3523, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1516, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-objecter_ops": { 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 1024, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:55.338 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 3523, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 3523, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 3523, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 3523, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:55.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-write_buf_throttle-0x556c4d8cc6e0": { 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 3758096384, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:55.341 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:55.342 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:55.984 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:55 smithi037 ceph-mon[29195]: pgmap v548: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.3 KiB/s wr, 15 op/s 2024-04-25T21:25:56.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:55 smithi098 ceph-mon[32569]: pgmap v548: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.3 KiB/s wr, 15 op/s 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:25:56.193 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:25:56.193 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T21:25:56.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T21:25:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:55 smithi177 ceph-mon[33043]: pgmap v548: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 2.3 KiB/s wr, 15 op/s 2024-04-25T21:25:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:25:58.159 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:57 smithi037 ceph-mon[29195]: pgmap v549: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:25:58.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:57 smithi098 ceph-mon[32569]: pgmap v549: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 362, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 349, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1632112, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 214624, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.066569646, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.024716200, 2024-04-25T21:25:58.200 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.039597826, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.007260271, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 349, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.018347148, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000052570 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1632112, 2024-04-25T21:25:58.201 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 214624 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 3, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 265, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 480, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 281216, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 6, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.022787606, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.016507801, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002490680, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:25:58.202 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 262, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.016920736, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000064582 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T21:25:58.203 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 281216 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000437134, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000362572, 2024-04-25T21:25:58.204 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:58.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:25:58.206 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:25:58.207 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-25T21:25:58.208 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-25T21:25:58.209 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-25T21:25:58.210 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 361, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1578516, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 361, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 361, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1578516, 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-25T21:25:58.211 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-25T21:25:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-25T21:25:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-25T21:25:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:58.212 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T21:25:58.212 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T21:25:58.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:57 smithi177 ceph-mon[33043]: pgmap v549: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:25:58.968 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T21:25:58.968 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T21:25:58.968 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T21:25:58.968 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T21:25:58.969 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T21:25:58.969 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:25:58.969 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:25:58.969 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T21:25:58.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T21:25:59.220 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:26:00.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:25:59 smithi098 ceph-mon[32569]: pgmap v550: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:26:00.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:25:59 smithi037 ceph-mon[29195]: pgmap v550: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:26:00.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:25:59 smithi177 ceph-mon[33043]: pgmap v550: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-25T21:26:01.188 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 45, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 54, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 11181, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 88997, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 19, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 6, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.010535932, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.000947093, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.006743752, 2024-04-25T21:26:01.189 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000523761, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 47, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.013201674, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000280886 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 32, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000003596, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000112 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3584, 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 4400 2024-04-25T21:26:01.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2146, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1746, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 566548, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1093093, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 10, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.165856068, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.072619807, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.097526864, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.007843534, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1748, 2024-04-25T21:26:01.191 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.085350880, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000048827 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3415, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000788557, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000230 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 619504 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 443, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 451, 2024-04-25T21:26:01.192 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1704107, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 240372, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.073967729, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.026297269, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.045172884, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.009321310, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 451, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.028634245, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000063490 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:01.193 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 56, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000012877, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000229 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1701296, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 230448 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:01.194 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 36, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001774008, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000049278 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000005656, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000002828 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-25T21:26:01.195 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 3, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 2, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000412064, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000206032 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 1, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-25T21:26:01.196 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 16, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 16, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 15, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 4, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 3, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 41, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 34, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 1, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 6, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:01.197 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 5, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 298, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2024-04-25T21:26:01.198 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 2, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:01.199 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 2, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 18, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 12, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 12, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-25T21:26:01.200 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 11, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 66, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:01.201 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 7, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 7, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-25T21:26:01.202 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4196347, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.010842957, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.001807159 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.203 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 20, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 22, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 2, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 19, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 23, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 4, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 16, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 20, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 4, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 2, 2024-04-25T21:26:01.204 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 2, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 55748, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 3, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 3, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 21, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 72, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.205 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000218359, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000218359 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000193705, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000193705 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-25T21:26:01.209 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.211 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.212 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.213 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.214 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-25T21:26:01.216 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 3, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 1, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 1, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1015, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:01.217 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:01.218 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-25T21:26:01.219 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 247847, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 81, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:01.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 82717, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 340, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-25T21:26:01.221 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 46, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 12670, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 29, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 46, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 46, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.209997522, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.004565163 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 46, 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 70 2024-04-25T21:26:01.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 46, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 46, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 6, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 9, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-25T21:26:01.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-25T21:26:01.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-25T21:26:01.225 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 113, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 14, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 5, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 17, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 2 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.226 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 76 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:01.227 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 2630, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2019812, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 2630, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2630, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2019812, 2024-04-25T21:26:01.228 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:01.229 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 46, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 12670, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 46, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 12670, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:01.230 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 46, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 46, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 46, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 46, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:01.231 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.232 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x56320cf3a500": { 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 7, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2043, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 7, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 6, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2043, 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:01.233 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:01.234 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:01.234 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:01.234 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.234 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:01.234 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-25T21:26:02.031 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T21:26:02.031 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T21:26:02.031 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=0 2024-04-25T21:26:02.032 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=18 2024-04-25T21:26:02.032 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T21:26:02.032 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:02.032 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:02.032 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T21:26:02.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T21:26:02.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:01 smithi098 ceph-mon[32569]: pgmap v551: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-04-25T21:26:02.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:01 smithi037 ceph-mon[29195]: pgmap v551: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-04-25T21:26:02.276 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:26:02.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:01 smithi177 ceph-mon[33043]: pgmap v551: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.5 KiB/s wr, 16 op/s 2024-04-25T21:26:04.003 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:03 smithi098 ceph-mon[32569]: pgmap v552: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 3630, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 3632, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1027298, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 1154347, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.281188412, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.089706954, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.068297085, 2024-04-25T21:26:04.004 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.100556139, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3630, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.114369094, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000031506 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.005 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 391, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 378, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1674572, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 219466, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 3, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.067247377, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.024362766, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.039548203, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.008957938, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 377, 2024-04-25T21:26:04.006 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.021203359, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000056242 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1674016, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 218768 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:04.007 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 8, 2024-04-25T21:26:04.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 268, 2024-04-25T21:26:04.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 3788, 2024-04-25T21:26:04.008 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 708974, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 8, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.016859645, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.010772993, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.003088121, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000031344, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 266, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.021274139, 2024-04-25T21:26:04.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000079977 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 708656 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-25T21:26:04.010 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 6098, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.141631732, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000023225 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:04.012 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 1, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-04-25T21:26:04.013 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:04.014 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:04.015 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-25T21:26:04.016 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-25T21:26:04.017 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-25T21:26:04.018 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 7, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-25T21:26:04.019 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4196347, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 4196206, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 7 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 3, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 3, 2024-04-25T21:26:04.020 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 1, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 1, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 50824, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 215416 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.021 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.022 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.024 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:04.025 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-25T21:26:04.026 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.027 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-25T21:26:04.028 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.029 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.030 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.033 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 3, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 2, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 3494, 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:04.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:04.035 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:04.036 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 182865, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 50, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-25T21:26:04.037 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 8512, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 13, 2024-04-25T21:26:04.038 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3633, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3633, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3633, 2024-04-25T21:26:04.039 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.928976994, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000530959 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3633, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3634 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3633, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 3633, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 2380, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-04-25T21:26:04.040 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 1251, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:04.041 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-25T21:26:04.042 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 113, 2024-04-25T21:26:04.043 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 25, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 4, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 5, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 1, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.044 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:04.045 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 4025, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2320379, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 4025, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4025, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2320379, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3633, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 2048, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3633, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2048, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3633, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3633, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3633, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3633, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.050 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x556cd2da26e0": { 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:04.051 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:04.052 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-25T21:26:04.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:03 smithi037 ceph-mon[29195]: pgmap v552: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T21:26:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:03 smithi177 ceph-mon[33043]: pgmap v552: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T21:26:04.920 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T21:26:04.920 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T21:26:04.920 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T21:26:04.920 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T21:26:04.921 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T21:26:04.921 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:04.921 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:04.921 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T21:26:04.921 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T21:26:05.160 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:26:06.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:05 smithi098 ceph-mon[32569]: pgmap v553: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:26:06.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:05 smithi037 ceph-mon[29195]: pgmap v553: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:26:06.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:05 smithi177 ceph-mon[33043]: pgmap v553: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-25T21:26:06.917 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-25T21:26:06.975 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:06.975 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 23, 2024-04-25T21:26:06.975 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 237, 2024-04-25T21:26:06.975 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 5326, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 611739, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 15, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 2, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.017059092, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.009066620, 2024-04-25T21:26:06.976 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.003977409, 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000422371, 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 235, 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.026749514, 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000113827 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.977 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:06.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-04-25T21:26:06.978 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000383, 2024-04-25T21:26:06.978 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000095 2024-04-25T21:26:06.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.978 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 555552 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2538, 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1878, 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 2248633, 2024-04-25T21:26:06.979 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 687909, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.211143852, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.082198833, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.134762921, 2024-04-25T21:26:06.980 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.016272889, 2024-04-25T21:26:06.981 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:06.981 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1877, 2024-04-25T21:26:06.981 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.078240180, 2024-04-25T21:26:06.981 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000041683 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3388, 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000818575, 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000241 2024-04-25T21:26:06.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.983 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1682096, 2024-04-25T21:26:06.983 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 222144 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 39, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 117, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 10863, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 197091, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 12, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:06.986 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.016121609, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.003920738, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.010614169, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.000554128, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 115, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.017652371, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000153498 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 66, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000015925, 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000241 2024-04-25T21:26:06.987 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 6496, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 165952 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:06.988 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 35, 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.001974001, 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000056400 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2, 2024-04-25T21:26:06.989 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000007451, 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000003725 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-25T21:26:06.990 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 11, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 16, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 16, 2024-04-25T21:26:06.991 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 15, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 4, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 3, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 36, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 30, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 6, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:06.992 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 5, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 298, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 6, 2024-04-25T21:26:06.993 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:06.994 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.995 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-25T21:26:06.996 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-25T21:26:06.996 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 18, 2024-04-25T21:26:06.996 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 12, 2024-04-25T21:26:06.996 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 12, 2024-04-25T21:26:06.996 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-25T21:26:06.997 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 11, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 66, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:06.998 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 5, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 5, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-25T21:26:06.999 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 2, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4195815, 2024-04-25T21:26:07.000 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 0, 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 4, 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.008628337, 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.002157084 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.001 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 0 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 20, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 22, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 2, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 19, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 23, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 4, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 16, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 20, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 4, 2024-04-25T21:26:07.002 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 52544, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 223608 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 17, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 72, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.003 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.004 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.005 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.006 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.008 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-25T21:26:07.008 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.008 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.009 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.010 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.011 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.012 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.013 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.014 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 1, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 2, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 1, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 1, 2024-04-25T21:26:07.016 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 1022, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:07.017 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:07.018 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:07.019 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 247751, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 81, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:07.020 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 82693, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 337, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.021 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 41, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 11903, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 28, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 41, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 41, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.213997479, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.005219450 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-25T21:26:07.022 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 41, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 63 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 41, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 0, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 41, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 0, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 10, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 0, 2024-04-25T21:26:07.023 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 4, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 8, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 6, 2024-04-25T21:26:07.024 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 20, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-25T21:26:07.025 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-25T21:26:07.026 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 114, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 26, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 9, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 12, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 3, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-25T21:26:07.027 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 14, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 0, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 1 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 1, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-25T21:26:07.028 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 76 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:07.029 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 2596, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2006284, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 2596, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 2596, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2006284, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.030 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:07.031 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 41, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 11903, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 41, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 11903, 2024-04-25T21:26:07.032 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.033 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-25T21:26:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:07.034 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 41, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 41, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 41, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 41, 2024-04-25T21:26:07.035 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.036 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:07.037 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:07.038 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:07.039 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55db263ac500": { 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 5, 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 1511, 2024-04-25T21:26:07.040 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:07.041 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 5, 2024-04-25T21:26:07.041 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:07.041 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:07.041 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 4, 2024-04-25T21:26:07.041 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 1511, 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.042 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:07.043 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=18 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:07.755 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:07.756 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T21:26:07.756 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T21:26:07.993 INFO:teuthology.orchestra.run.smithi098.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:26:08.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:07 smithi098 ceph-mon[32569]: pgmap v554: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-25T21:26:08.115 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:07 smithi098 ceph-mon[32569]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T21:26:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:07 smithi037 ceph-mon[29195]: pgmap v554: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-25T21:26:08.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:07 smithi037 ceph-mon[29195]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T21:26:08.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:07 smithi177 ceph-mon[33043]: pgmap v554: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-25T21:26:08.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:07 smithi177 ceph-mon[33043]: osdmap e114: 12 total, 12 up, 12 in 2024-04-25T21:26:09.736 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-04-25T21:26:09.736 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:09.736 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 396, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 383, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 1677074, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 222468, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 6, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 1, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.068477034, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.023065506, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.043210809, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.009067998, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 382, 2024-04-25T21:26:09.737 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.019467372, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000050961 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 1676240, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 222144 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 1064, 2024-04-25T21:26:09.738 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 1330, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 261594, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 962041, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 7, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.116218608, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.043752014, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.022696905, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.036525347, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 1327, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.060719581, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000045757 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.739 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 624320 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_messages": 2508, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages": 2508, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_bytes": 753762, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_bytes": 796778, 2024-04-25T21:26:09.740 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_created_connections": 4, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_active_connections": 3, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_total_time": 0.264688372, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_send_time": 0.094884611, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_recv_time": 0.064362443, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_running_fast_dispatch_time": 0.080553450, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 2507, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.111384093, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000044429 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.741 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "cct": { 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "total_workers": 1, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "unhealthy_workers": 0 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-MDSRank": { 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:09.742 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 5973, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.177176193, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000029662 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "queue_len": 0, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "complete_latency": { 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": { 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "request": 0, 2024-04-25T21:26:09.743 INFO:teuthology.orchestra.run.smithi098.stdout: "reply": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "reply_latency": { 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_reply": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "forward": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_commit": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_split": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_merge": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes": 0, 2024-04-25T21:26:09.744 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_top": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_bottom": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_pinned": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_expired": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "subtrees": 1, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_hit": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_forward": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_discover": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "traverse_lock": 0, 2024-04-25T21:26:09.745 INFO:teuthology.orchestra.run.smithi098.stdout: "load_cent": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "q": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "exported": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "exported_inodes": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "imported": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "imported_inodes": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rfiles": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rbytes": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "root_rsnaps": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:09.747 INFO:teuthology.orchestra.run.smithi098.stdout: "process_request_cap_release": 0 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_cache": { 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_update_receipt": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_try_discover": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_send_discover": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_created": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_enqueued": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "strays_migrated": 0, 2024-04-25T21:26:09.748 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_started": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "recovery_completed": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_flush": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "ireq_inodestats": 0 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_log": { 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "evlrg": 0, 2024-04-25T21:26:09.749 INFO:teuthology.orchestra.run.smithi098.stdout: "evadd": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "evex": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "evtrm": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "ev": 8, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "evexg": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "evexd": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segadd": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segex": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segtrm": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "seg": 2, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segmjr": 2, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segexg": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "segexd": 0, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "expos": 4194304, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "wrpos": 4196655, 2024-04-25T21:26:09.750 INFO:teuthology.orchestra.run.smithi098.stdout: "rdpos": 4196514, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "jlat": { 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "replayed": 8 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_mem": { 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "ino": 3, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "ino+": 3, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "ino-": 0, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "dir": 1, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "dir+": 1, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "dir-": 0, 2024-04-25T21:26:09.751 INFO:teuthology.orchestra.run.smithi098.stdout: "dn": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "dn+": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "dn-": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "cap": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "cap+": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "cap-": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "rss": 53312, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "heap": 215416 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_server": { 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_client_request": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_request": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_client_session": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "handle_peer_request": 0, 2024-04-25T21:26:09.752 INFO:teuthology.orchestra.run.smithi098.stdout: "req_create_latency": { 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getattr_latency": { 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.753 INFO:teuthology.orchestra.run.smithi098.stdout: "req_link_latency": { 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookup_latency": { 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.754 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupino_latency": { 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.755 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupname_latency": { 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.756 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "req_lssnap_latency": { 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mkdir_latency": { 2024-04-25T21:26:09.757 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mknod_latency": { 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.758 INFO:teuthology.orchestra.run.smithi098.stdout: "req_mksnap_latency": { 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "req_open_latency": { 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "req_readdir_latency": { 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.759 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rename_latency": { 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmdir_latency": { 2024-04-25T21:26:09.760 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.761 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setattr_latency": { 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setlayout_latency": { 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "req_setxattr_latency": { 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "req_symlink_latency": { 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "req_unlink_latency": { 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_sessions": { 2024-04-25T21:26:09.764 INFO:teuthology.orchestra.run.smithi098.stdout: "session_count": 2, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "session_add": 4, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "session_remove": 2, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_open": 4, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "sessions_stale": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "total_load": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "average_load": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "avg_session_uptime": 3502, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "mempool": { 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:09.765 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:09.766 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_items": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:09.767 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_bytes": 190951, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_anon_items": 54, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_bytes": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_items": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:09.768 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_items": 167, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "pgmap_items": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_bytes": 8512, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_co_items": 13, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_1_items": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "unittest_2_items": 0 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.769 INFO:teuthology.orchestra.run.smithi098.stdout: "objecter": { 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_active": 0, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_laggy": 0, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send": 3567, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_send_bytes": 21, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_resend": 3, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_reply": 3567, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_latency": { 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3567, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 1.597980970, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000447990 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "op_inflight": 0, 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "oplen_avg": { 2024-04-25T21:26:09.770 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 3567, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 3568 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "op": 3567, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "op_r": 3567, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "op_w": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "op_rmw": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "op_pg": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_stat": 2308, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_create": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_read": 1257, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_write": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writefull": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_writesame": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_append": 0, 2024-04-25T21:26:09.771 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_zero": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_truncate": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_delete": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_mapext": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_getxattr": 1, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_setxattr": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_call": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_watch": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_notify": 0, 2024-04-25T21:26:09.772 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "osdop_other": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_active": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_send": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_resend": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "linger_ping": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_active": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_send": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolop_resend": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_active": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_send": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "poolstat_resend": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_active": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_send": 0, 2024-04-25T21:26:09.773 INFO:teuthology.orchestra.run.smithi098.stdout: "statfs_resend": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "command_active": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "command_send": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "command_resend": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "map_epoch": 114, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "map_full": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "map_inc": 26, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_sessions": 5, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_open": 6, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_session_close": 1, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_laggy": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_wr": 0, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_rd": 2, 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_del": 0 2024-04-25T21:26:09.774 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "oft": { 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_objs": 0, 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_updates": 0, 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "omap_total_removes": 0 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.775 INFO:teuthology.orchestra.run.smithi098.stdout: "osd-slow-ops": { 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "slow_ops_count": 0 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "purge_queue": { 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:09.776 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing": 0, 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_executed": 0, 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.777 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 3964, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 2312373, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 3964, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3964, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2312373, 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:09.778 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:09.779 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 104857600, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:09.780 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3567, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 2356, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3567, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 2356, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.781 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 1024, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:09.782 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 3567, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 3567, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 3567, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 3567, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.783 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:09.784 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "throttle-write_buf_throttle-0x55d1645746e0": { 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "val": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "max": 3758096384, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get_started": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get": 0, 2024-04-25T21:26:09.785 INFO:teuthology.orchestra.run.smithi098.stdout: "get_sum": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "take": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "take_sum": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "put": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "put_sum": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "wait": { 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "avgcount": 0, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "sum": 0.000000000, 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: "avgtime": 0.000000000 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-04-25T21:26:09.786 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-04-25T21:26:10.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:09 smithi098 ceph-mon[32569]: pgmap v556: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:26:10.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:09 smithi037 ceph-mon[29195]: pgmap v556: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:26:10.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:09 smithi177 ceph-mon[33043]: pgmap v556: 97 pgs: 97 active+clean; 5.2 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T21:26:10.545 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:10.546 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:10.546 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T21:26:10.546 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T21:26:10.797 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:26:12.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:11 smithi098 ceph-mon[32569]: pgmap v557: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:12.170 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:11 smithi177 ceph-mon[33043]: pgmap v557: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:12.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:11 smithi037 ceph-mon[29195]: pgmap v557: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2136, 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1587, 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 567100, 2024-04-25T21:26:12.587 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 678860, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 9, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.190295858, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.092348254, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.106702162, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.006788274, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1590, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.088590177, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000055717 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3439, 2024-04-25T21:26:12.588 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000863718, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000251 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 7376, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 21520 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 35, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 88, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 9416, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 199246, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 15, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 7, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.009621598, 2024-04-25T21:26:12.589 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.002964065, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.005098657, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000320538, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 85, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.016610922, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000195422 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 33, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000004577, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000138 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.590 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 4464, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 161872 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 453, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 712, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1700754, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 850164, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 13, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.100335896, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.038166483, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.058297940, 2024-04-25T21:26:12.591 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.010656405, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 707, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.042455965, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000060050 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 92, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000023548, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000255 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 1698784, 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 800128 2024-04-25T21:26:12.592 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 48, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001399315, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000029152 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:12.593 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000005367, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000002683 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 10, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 7, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 7, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001229645, 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000175663 2024-04-25T21:26:12.594 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 3, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 16, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 16, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 15, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 4, 2024-04-25T21:26:12.595 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 2, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 2, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 3, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 49, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 42, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 6, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-04-25T21:26:12.596 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 5, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 298, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 6, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:12.597 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 5, 2024-04-25T21:26:12.598 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 5, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 18, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 12, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 12, 2024-04-25T21:26:12.599 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-04-25T21:26:12.600 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 11, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 66, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 8, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 8, 2024-04-25T21:26:12.601 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4196655, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 7, 2024-04-25T21:26:12.602 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.014009237, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002001319 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 20, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 22, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 2, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 19, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 23, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 4, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 16, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 20, 2024-04-25T21:26:12.603 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 4, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 2, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 5, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 3, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 52624, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223608 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 10, 2024-04-25T21:26:12.604 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 10, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 35, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 72, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000784089, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000196022 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:12.605 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000321804, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000160902 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.606 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000123752, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000123752 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:12.607 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.608 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-04-25T21:26:12.609 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.610 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.611 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:12.613 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-04-25T21:26:12.614 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.615 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 2, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 4, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 2, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 2, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 5, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 1, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 1033, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:12.616 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:12.617 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:12.618 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 256087, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 84, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:12.619 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8152, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 170, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 83469, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 346, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-04-25T21:26:12.620 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 47, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 13159, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 33, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 47, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 47, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.227993687, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004850929 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 47, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 72 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 47, 2024-04-25T21:26:12.621 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 47, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 7, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 8, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-04-25T21:26:12.622 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-04-25T21:26:12.623 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-04-25T21:26:12.624 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 114, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 26, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 8, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 13, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 5, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 20, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 1 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-04-25T21:26:12.625 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 76 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:12.626 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2620, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2014538, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2620, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2620, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2014538, 2024-04-25T21:26:12.627 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:12.628 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 47, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 13159, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 47, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 13159, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-04-25T21:26:12.629 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 47, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 47, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 47, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 47, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.630 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.631 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x55c6e65aa500": { 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 8, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2351, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 8, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 7, 2024-04-25T21:26:12.632 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2351, 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:12.633 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=18 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T21:26:13.445 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:13.446 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:13.446 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T21:26:13.446 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T21:26:13.691 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:26:14.055 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:13 smithi177 ceph-mon[33043]: pgmap v558: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:14.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:13 smithi098 ceph-mon[32569]: pgmap v558: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:14.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:13 smithi037 ceph-mon[29195]: pgmap v558: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2112, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1513, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 560050, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 538733, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 11, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2024-04-25T21:26:15.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.187774996, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.090477922, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.105641136, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.006693938, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1515, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.082571172, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000054502 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3335, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001218094, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000365 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 442, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 427, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1694955, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 232311, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 8, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.072785850, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.024749183, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.046188539, 2024-04-25T21:26:15.402 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.009651088, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 426, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.031104183, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000073014 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000001256, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000157 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 1691664, 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 226640 2024-04-25T21:26:15.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 45, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 343, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 12180, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 696692, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 17, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.033783927, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.015913318, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.014079920, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000180582, 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:15.404 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 337, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.027193292, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000080692 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 91, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000025278, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000277 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 10000, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 640608 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-04-25T21:26:15.405 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 41, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002417014, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000058951 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000004488, 2024-04-25T21:26:15.406 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000002244 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 5, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 3, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001840286, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000613428 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 2, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:15.407 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 11, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 16, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 16, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 15, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 5, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 3, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 45, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 34, 2024-04-25T21:26:15.408 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 2, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 7, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:15.409 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 5, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 298, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 6, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 2, 2024-04-25T21:26:15.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 2, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 18, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 12, 2024-04-25T21:26:15.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 13, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-04-25T21:26:15.412 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 11, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 66, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-04-25T21:26:15.413 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 9, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 9, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 2, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 2, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-04-25T21:26:15.414 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 4196796, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.013541243, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001692655 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 20, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 22, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 2, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 19, 2024-04-25T21:26:15.415 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 23, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 4, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 16, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 21, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 5, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 2, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 2, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 53648, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 223608 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 9, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:15.416 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 5, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 26, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 73, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001347888, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001347888 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.417 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000492398, 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000246199 2024-04-25T21:26:15.418 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-04-25T21:26:15.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.422 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:15.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.424 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.426 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 4, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 3, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 2, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 1066, 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:15.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:15.429 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:15.430 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 239415, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 73, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:15.431 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8152, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 170, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 82781, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 348, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 47, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 13029, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 33, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 47, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 47, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.220993878, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004701997 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.433 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 47, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 71 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 47, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 47, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 8, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 8, 2024-04-25T21:26:15.434 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 6, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:15.435 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-04-25T21:26:15.436 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 114, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 26, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 8, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 13, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 5, 2024-04-25T21:26:15.437 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 17, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 2 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 76 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.438 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2595, 2024-04-25T21:26:15.439 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2007500, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2595, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2595, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2007500, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:15.440 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 47, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 13029, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 47, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 13029, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.441 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 47, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 47, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 47, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 47, 2024-04-25T21:26:15.442 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:15.443 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x5608039d6500": { 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 9, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2492, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:15.444 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 9, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 8, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2492, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:15.445 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-04-25T21:26:16.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:15 smithi098 ceph-mon[32569]: pgmap v559: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:16.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:15 smithi098 ceph-mon[32569]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T21:26:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:15 smithi037 ceph-mon[29195]: pgmap v559: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:16.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:15 smithi037 ceph-mon[29195]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T21:26:16.263 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:15 smithi177 ceph-mon[33043]: pgmap v559: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:16.263 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:15 smithi177 ceph-mon[33043]: osdmap e115: 12 total, 12 up, 12 in 2024-04-25T21:26:16.292 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T21:26:16.292 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T21:26:16.292 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T21:26:16.292 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=18 2024-04-25T21:26:16.293 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-25T21:26:16.293 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:16.293 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:16.293 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T21:26:16.293 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T21:26:16.533 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:26:18.167 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:17 smithi098 ceph-mon[32569]: pgmap v561: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 323 B/s wr, 11 op/s 2024-04-25T21:26:18.176 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:17 smithi177 ceph-mon[33043]: pgmap v561: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 323 B/s wr, 11 op/s 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 5526, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 6985, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1748888, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 15699373, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 24, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.568545747, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.225281511, 2024-04-25T21:26:18.200 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.194451403, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.121069686, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 6986, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.336255341, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000048132 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 6864, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001706143, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000248 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 474752 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:18.201 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 5283, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 7285, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 1860691, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 11478686, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 13, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 6, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.674527358, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.271113778, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.275234611, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.131875465, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 7283, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.369109728, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000050681 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.202 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8586, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002010990, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000234 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 1024400, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 1706768 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 4567, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 3964, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3141177, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 8543384, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 211, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 7, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.581474360, 2024-04-25T21:26:18.203 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.160273007, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.270712875, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.146237686, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3960, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.341692241, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000086285 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5651, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001627232, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000287 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 2414704, 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 445152 2024-04-25T21:26:18.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 6537, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2.036303777, 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000311504 2024-04-25T21:26:18.205 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 16, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000251235, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000015702 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 878, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 878, 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-04-25T21:26:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 878, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 13.303741946, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.015152325 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 498, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 48, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 17, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 31, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:18.207 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 23, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 2, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 3, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 2, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 958, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 946, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-04-25T21:26:18.208 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 4, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 234, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 6, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:18.209 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 663, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 657, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 599, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 730, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 668, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 3, 2024-04-25T21:26:18.210 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 628 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 24, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 49, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 2, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 3, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 1, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:18.211 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:18.213 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:18.213 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-04-25T21:26:18.213 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 11, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 152, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 2648, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 2626, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 2626, 2024-04-25T21:26:18.214 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 22, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 41, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 85, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 84, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 84, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 1, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 1, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 1, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 29166407, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 29250803, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-04-25T21:26:18.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2557, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 5.622423947, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002198836 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 40, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 41, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 1, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 20, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 20, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 0, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 48, 2024-04-25T21:26:18.216 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 48, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 3, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 19, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 16, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 60996, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 231800 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 2161, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 289, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 878, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 45, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 554, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 15, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.013851178, 2024-04-25T21:26:18.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000923411 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 186, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.035801477, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000192481 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-04-25T21:26:18.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.001533321, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000383330 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000150157, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000150157 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.220 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 18, 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002727293, 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000151516 2024-04-25T21:26:18.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 9.672400427, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 1.209050053 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 18, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.091933892, 2024-04-25T21:26:18.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005107438 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000207091, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000207091 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.005131274, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002565637 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003244021, 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001622010 2024-04-25T21:26:18.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.014944708, 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.007472354 2024-04-25T21:26:18.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 12, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.057674248, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004806187 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 600, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3.379765547, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005632942 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.225 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.226 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.020989228, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002623653 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003388084, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003388084 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:18.227 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 2, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 6, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 4, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 2, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 80, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 40, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 1052, 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:18.228 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:18.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:18.230 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 353022, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 144, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 88, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 1, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-04-25T21:26:18.231 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 167, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 189059, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 1454, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-04-25T21:26:18.232 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 4650, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 26824792, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 32, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 4650, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4650, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 27.742229493, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.005966070 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-04-25T21:26:18.233 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4650, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 5690 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 4650, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 1, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 4649, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 478, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 29, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 1, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 2558, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 270, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-04-25T21:26:18.234 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 523, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 58, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:18.235 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 599, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-04-25T21:26:18.236 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 115, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 27, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 12, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 13, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 1, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 1158, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 16 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:18.237 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 424 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 1, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 1, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 1, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 1, 2024-04-25T21:26:18.238 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 15372, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 5152243, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 15372, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:18.239 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 15372, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 5152243, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:18.240 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4650, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 26824941, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4650, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 26824941, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-04-25T21:26:18.241 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4650, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 4650, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4650, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 4650, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.242 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 149, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 149, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.243 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x557d4a944500": { 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2648, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 25056499, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2648, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2557, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 25056499, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:18.244 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:18.245 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:18.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.245 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:18.245 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-04-25T21:26:18.245 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:17 smithi037 ceph-mon[29195]: pgmap v561: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 323 B/s wr, 11 op/s 2024-04-25T21:26:19.044 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T21:26:19.044 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T21:26:19.044 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=24 2024-04-25T21:26:19.044 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-25T21:26:19.044 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T21:26:19.045 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=18 2024-04-25T21:26:19.045 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=12 2024-04-25T21:26:19.045 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T21:26:19.045 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T21:26:19.283 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:26:20.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:19 smithi177 ceph-mon[33043]: pgmap v562: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:20.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:19 smithi037 ceph-mon[29195]: pgmap v562: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:20.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:19 smithi098 ceph-mon[32569]: pgmap v562: 97 pgs: 97 active+clean; 5.2 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 2776, 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 2761, 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 10777654, 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 979148, 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2024-04-25T21:26:20.880 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.329368006, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.098410255, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.122146800, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.093241172, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2761, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.118738083, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000043005 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.881 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 1680720, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 224848 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 880, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1149, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 8609838, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 994856, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 8, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.104293320, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.038366027, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.029473098, 2024-04-25T21:26:20.882 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.026402633, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1146, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.055493923, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000048424 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 715904 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.883 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 1216, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 1218, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 8584061, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 386695, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 6, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.119317290, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.039171690, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.033416687, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.035886725, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1216, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.043191308, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000035519 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.884 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:20.885 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 7570, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.222685575, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000029416 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.886 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2024-04-25T21:26:20.887 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 36, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 16, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 20, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 9, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 2, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2024-04-25T21:26:20.888 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 4, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 234, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 6, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:26:20.889 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2024-04-25T21:26:20.890 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2024-04-25T21:26:20.891 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "evlrg": 0, 2024-04-25T21:26:20.892 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 2648, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segmjr": 80, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 29166407, 2024-04-25T21:26:20.893 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 29250803, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 29250691, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 2648 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 31, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 32, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 1, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 11, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 11, 2024-04-25T21:26:20.894 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 36, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 36, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 52500, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 215416 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2024-04-25T21:26:20.895 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2024-04-25T21:26:20.896 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.897 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.898 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.899 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.900 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.901 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.902 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.903 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.904 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.905 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.906 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.907 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 2, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 6, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 4, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 5, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 4150, 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: "mdthresh_evicted": 0 2024-04-25T21:26:20.908 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:26:20.909 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:26:20.910 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 200076, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 87, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:26:20.911 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8032, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 167, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 144641, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 1128, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2024-04-25T21:26:20.912 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 4465, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 26, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 4, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 4465, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4465, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2.325935018, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000520926 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2024-04-25T21:26:20.913 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4465, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 4466 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 4465, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 4465, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 2509, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 1953, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2024-04-25T21:26:20.914 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 2, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2024-04-25T21:26:20.915 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2024-04-25T21:26:20.916 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 115, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 27, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 6, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 7, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 1, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2024-04-25T21:26:20.917 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.918 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed_ops": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 4868, 2024-04-25T21:26:20.919 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 27509849, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 4868, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4868, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 27509849, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2024-04-25T21:26:20.920 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4465, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 25056509, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4465, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 25056509, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.921 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 4465, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 4465, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 4465, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 4465, 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:20.922 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-04-25T21:26:20.923 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x561a9a3966e0": { 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2024-04-25T21:26:20.924 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout: } 2024-04-25T21:26:20.925 INFO:teuthology.orchestra.run.smithi177.stdout:} 2024-04-25T21:26:21.738 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T21:26:21.738 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T21:26:21.738 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T21:26:21.738 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-25T21:26:21.739 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-25T21:26:21.739 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:26:21.739 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:26:21.739 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T21:26:21.751 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T21:26:21.751 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T21:26:21.751 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:26:21.773 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T21:26:21.773 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:26:21.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T21:26:22.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:21 smithi177 ceph-mon[33043]: pgmap v563: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:22.134 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:22.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:21 smithi037 ceph-mon[29195]: pgmap v563: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:22.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:21 smithi098 ceph-mon[32569]: pgmap v563: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 -- 172.21.15.37:0/2302479943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95401022f0 msgr2=0x7f954010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 --2- 172.21.15.37:0/2302479943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95401022f0 0x7f954010e7e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f953800b0a0 tx=0x7f953800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 -- 172.21.15.37:0/2302479943 shutdown_connections 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 --2- 172.21.15.37:0/2302479943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95401022f0 0x7f954010e7e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 --2- 172.21.15.37:0/2302479943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540107980 0x7f9540107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 --2- 172.21.15.37:0/2302479943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9540101930 0x7f9540101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 -- 172.21.15.37:0/2302479943 >> 172.21.15.37:0/2302479943 conn(0x7f95400788b0 msgr2=0x7f9540078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 -- 172.21.15.37:0/2302479943 shutdown_connections 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.000+0000 7f9547a71700 1 -- 172.21.15.37:0/2302479943 wait complete. 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 Processor -- start 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 -- start start 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 0x7f95401034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 0x7f9540103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9540111ad0 con 0x7f9540107980 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9540111950 con 0x7f95401022f0 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f9547a71700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f9540111c50 con 0x7f9540101930 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57586/0 (socket says 172.21.15.37:57586) 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 -- 172.21.15.37:0/4231181730 learned_addr learned my addr 172.21.15.37:0/4231181730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954500c700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 0x7f9540103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:24.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954580d700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 0x7f95401034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 -- 172.21.15.37:0/4231181730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 msgr2=0x7f95401034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 0x7f95401034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 -- 172.21.15.37:0/4231181730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 msgr2=0x7f9540103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 0x7f9540103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954600e700 1 -- 172.21.15.37:0/4231181730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a75e0 con 0x7f9540107980 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954580d700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 0x7f95401034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.001+0000 7f954500c700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 0x7f9540103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f954600e700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f95380062a0 tx=0x7f9538004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538021070 con 0x7f9540107980 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a7870 con 0x7f9540107980 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9538003c00 con 0x7f9540107980 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538004830 con 0x7f9540107980 2024-04-25T21:26:24.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.002+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a7d70 con 0x7f9540107980 2024-04-25T21:26:24.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.003+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540069730 con 0x7f9540107980 2024-04-25T21:26:24.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.004+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9538007a00 con 0x7f9540107980 2024-04-25T21:26:24.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.005+0000 7f9536ffd700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 0x7f952c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:24.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.005+0000 7f954580d700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 0x7f952c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:24.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.005+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f953800f020 con 0x7f9540107980 2024-04-25T21:26:24.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.005+0000 7f954580d700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 0x7f952c0a6270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f953c005e90 tx=0x7f953c005d20 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:24.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.009+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f953802a020 con 0x7f9540107980 2024-04-25T21:26:24.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.233+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f954002df80 con 0x7f9540107980 2024-04-25T21:26:24.233 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:23 smithi037 ceph-mon[29195]: pgmap v564: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.233+0000 7f9536ffd700 1 -- 172.21.15.37:0/4231181730 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v115) ==== 81+0+944 (secure 0 0 0) 0x7f953809e450 con 0x7f9540107980 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2407888079 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1634118111 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3727603400 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/234140491 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2407888079 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/3898721946 2024-04-26T20:57:34.092925+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/3898721946 2024-04-26T20:57:34.092925+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4006052172 2024-04-26T20:57:34.092925+0000 2024-04-25T21:26:24.234 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1653551922 2024-04-26T20:57:06.373285+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1698545333 2024-04-26T20:57:34.092925+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/767353771 2024-04-26T20:57:06.373285+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3184535068 2024-04-26T20:57:34.092925+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1398754476 2024-04-26T21:08:28.094244+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1093601384 2024-04-26T20:57:06.373285+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1828557826 2024-04-26T20:57:06.373285+0000 2024-04-25T21:26:24.235 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1093601384 2024-04-26T20:57:06.373285+0000 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 msgr2=0x7f952c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 0x7f952c0a6270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f953c005e90 tx=0x7f953c005d20 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 msgr2=0x7f9540103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f95380062a0 tx=0x7f9538004d30 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 shutdown_connections 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f952c0a3db0 0x7f952c0a6270 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9540107980 0x7f9540103f60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9540101930 0x7f95401034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 --2- 172.21.15.37:0/4231181730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95401022f0 0x7f9540103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 >> 172.21.15.37:0/4231181730 conn(0x7f95400788b0 msgr2=0x7f954006b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:24.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 shutdown_connections 2024-04-25T21:26:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:24.236+0000 7f9547a71700 1 -- 172.21.15.37:0/4231181730 wait complete. 2024-04-25T21:26:24.237 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-04-25T21:26:24.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:23 smithi177 ceph-mon[33043]: pgmap v564: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:24.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:23 smithi098 ceph-mon[32569]: pgmap v564: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T21:26:25.013 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T21:26:25.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:26:25.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T21:26:25.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:26:25.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:26:25.057 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T21:26:25.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:26:25.121 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T21:26:25.176 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T21:26:25.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:26:25.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T21:26:25.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:24 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/4231181730' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:26:25.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:26:25.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T21:26:25.329 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T21:26:25.341 INFO:tasks.cephadm:Teardown begin 2024-04-25T21:26:25.341 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T21:26:25.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:26:25.342 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:24 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/4231181730' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:26:25.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:24 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/4231181730' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:26:25.595 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:26.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:25 smithi037 ceph-mon[29195]: pgmap v565: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:26:26.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:25 smithi177 ceph-mon[33043]: pgmap v565: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:26:26.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:25 smithi098 ceph-mon[32569]: pgmap v565: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 -- 172.21.15.37:0/1054345810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 msgr2=0x7f95b8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/1054345810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b8108e40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009990 tx=0x7f95a8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 -- 172.21.15.37:0/1054345810 shutdown_connections 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/1054345810 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/1054345810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b8108e40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/1054345810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 0x7f95b806b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 -- 172.21.15.37:0/1054345810 >> 172.21.15.37:0/1054345810 conn(0x7f95b80789e0 msgr2=0x7f95b8078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 -- 172.21.15.37:0/1054345810 shutdown_connections 2024-04-25T21:26:27.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.323+0000 7f95bd5d8700 1 -- 172.21.15.37:0/1054345810 wait complete. 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 Processor -- start 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 -- start start 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 0x7f95b8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8112c60 con 0x7f95b8108a60 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8112ae0 con 0x7f95b806b1f0 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95bd5d8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8112de0 con 0x7f95b806bbb0 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.324+0000 7f95b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57610/0 (socket says 172.21.15.37:57610) 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 -- 172.21.15.37:0/2535740181 learned_addr learned my addr 172.21.15.37:0/2535740181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b6ffd700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 0x7f95b8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:27.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b67fc700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 -- 172.21.15.37:0/2535740181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 msgr2=0x7f95b81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 -- 172.21.15.37:0/2535740181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 msgr2=0x7f95b8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 0x7f95b8199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 -- 172.21.15.37:0/2535740181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b819e150 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b67fc700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95b77fe700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f95b0007ee0 tx=0x7f95b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b000dbd0 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b819e440 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b000dd70 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0005990 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.325+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b81a5cc0 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.327+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b810b060 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.328+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f95b001a020 con 0x7f95b8108a60 2024-04-25T21:26:27.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.328+0000 7f959ffff700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 0x7f95a00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:27.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.328+0000 7f95b6ffd700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 0x7f95a00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:27.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.329+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f95b00cff20 con 0x7f95b8108a60 2024-04-25T21:26:27.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.329+0000 7f95b6ffd700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 0x7f95a00a6270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b5c0 tx=0x7f95a8005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:27.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.333+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95b0010070 con 0x7f95b8108a60 2024-04-25T21:26:27.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.588+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95b8069730 con 0x7f95b8108a60 2024-04-25T21:26:27.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.589+0000 7f959ffff700 1 -- 172.21.15.37:0/2535740181 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7f95b009c5c0 con 0x7f95b8108a60 2024-04-25T21:26:27.589 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:26:27.589 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:26:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 msgr2=0x7f95a00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:27.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 0x7f95a00a6270 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f95a800b5c0 tx=0x7f95a8005960 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 msgr2=0x7f95b819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f95b0007ee0 tx=0x7f95b000d3b0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 shutdown_connections 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f95a00a3db0 0x7f95a00a6270 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95b8108a60 0x7f95b819da10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95b806b1f0 0x7f95b8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 --2- 172.21.15.37:0/2535740181 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95b806bbb0 0x7f95b81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 >> 172.21.15.37:0/2535740181 conn(0x7f95b80789e0 msgr2=0x7f95b80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 shutdown_connections 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:27.591+0000 7f95bd5d8700 1 -- 172.21.15.37:0/2535740181 wait complete. 2024-04-25T21:26:27.592 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:26:28.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:27 smithi037 ceph-mon[29195]: pgmap v566: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:26:28.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:27 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2535740181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:28.248 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:26:28.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:27 smithi177 ceph-mon[33043]: pgmap v566: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:26:28.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:27 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2535740181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:28.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:27 smithi098 ceph-mon[32569]: pgmap v566: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T21:26:28.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:27 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2535740181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:28.505 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:30.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:29 smithi037 ceph-mon[29195]: pgmap v567: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:26:30.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 -- 172.21.15.37:0/192741663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 msgr2=0x7fb708111970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:30.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/192741663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb708111970 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b0a0 tx=0x7fb70000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 -- 172.21.15.37:0/192741663 shutdown_connections 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/192741663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb708111970 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/192741663 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 0x7fb70806b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/192741663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 0x7fb70810f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 -- 172.21.15.37:0/192741663 >> 172.21.15.37:0/192741663 conn(0x7fb708077990 msgr2=0x7fb708077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 -- 172.21.15.37:0/192741663 shutdown_connections 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.227+0000 7fb70cf5f700 1 -- 172.21.15.37:0/192741663 wait complete. 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 Processor -- start 2024-04-25T21:26:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 -- start start 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 0x7fb70819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 0x7fb70819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114e90 con 0x7fb70810f580 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb708114d10 con 0x7fb70806b1b0 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70cf5f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7fb708115010 con 0x7fb70806bad0 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb706d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb706d9d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37300/0 (socket says 172.21.15.37:37300) 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb706d9d700 1 -- 172.21.15.37:0/3946837829 learned_addr learned my addr 172.21.15.37:0/3946837829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb705d9b700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 0x7fb70819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.228+0000 7fb70659c700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 0x7fb70819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 -- 172.21.15.37:0/3946837829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 msgr2=0x7fb70819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 0x7fb70819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 -- 172.21.15.37:0/3946837829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 msgr2=0x7fb70819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 0x7fb70819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 -- 172.21.15.37:0/3946837829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a0330 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb706d9d700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b070 tx=0x7fb700004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700021070 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb700007e30 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a05c0 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7000046f0 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.229+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a7ea0 con 0x7fb70810f580 2024-04-25T21:26:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.230+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb708101fb0 con 0x7fb70810f580 2024-04-25T21:26:30.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.231+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb700007980 con 0x7fb70810f580 2024-04-25T21:26:30.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.232+0000 7fb6f77fe700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 0x7fb6f00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:30.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.232+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7fb7000d1890 con 0x7fb70810f580 2024-04-25T21:26:30.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.232+0000 7fb70659c700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 0x7fb6f00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:30.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.232+0000 7fb70659c700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 0x7fb6f00a6270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005950 tx=0x7fb6fc00a300 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.237+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb70009dfc0 con 0x7fb70810f580 2024-04-25T21:26:30.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:29 smithi177 ceph-mon[33043]: pgmap v567: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:26:30.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:29 smithi098 ceph-mon[32569]: pgmap v567: 97 pgs: 97 active+clean; 5.2 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T21:26:30.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.491+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb70802df80 con 0x7fb70810f580 2024-04-25T21:26:30.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.492+0000 7fb6f77fe700 1 -- 172.21.15.37:0/3946837829 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11923 (secure 0 0 0) 0x7fb70009d8c0 con 0x7fb70810f580 2024-04-25T21:26:30.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:26:30.492 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14508,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/719027051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":719027051},{"type":"v1","addr":"172.21.15.37:6835","nonce":719027051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/1767596053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":1767596053},{"type":"v1","addr":"172.21.15.37:6837","nonce":1767596053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T21:05:03.820512+0000","modified":"2024-04-25T21:08:26.488484+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24469,"mds_1":24458,"mds_2":24457,"mds_3":24445,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14568":{"gid":14568,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6839/2484283883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2484283883},{"type":"v1","addr":"172.21.15.37:6839","nonce":2484283883}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.37:6841/2041063205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2041063205},{"type":"v1","addr":"172.21.15.37:6841","nonce":2041063205}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6833/3425764534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3425764534},{"type":"v1","addr":"172.21.15.98:6833","nonce":3425764534}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.98:6835/785792135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":785792135},{"type":"v1","addr":"172.21.15.98:6835","nonce":785792135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6833/3715259407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":3715259407},{"type":"v1","addr":"172.21.15.177:6833","nonce":3715259407}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.98:6837/2446670971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2446670971},{"type":"v1","addr":"172.21.15.98:6837","nonce":2446670971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.177:6835/724905952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":724905952},{"type":"v1","addr":"172.21.15.177:6835","nonce":724905952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6837/2828024557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":2828024557},{"type":"v1","addr":"172.21.15.177:6837","nonce":2828024557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"b","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.98:6839/899178427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6838","nonce":899178427},{"type":"v1","addr":"172.21.15.98:6839","nonce":899178427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.177:6839/3063347892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":3063347892},{"type":"v1","addr":"172.21.15.177:6839","nonce":3063347892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24469,"qdb_cluster":[24419,24445,24457,24458,24469]},"id":1}]} 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 msgr2=0x7fb6f00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 0x7fb6f00a6270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc005950 tx=0x7fb6fc00a300 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 msgr2=0x7fb70819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b070 tx=0x7fb700004dc0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 shutdown_connections 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7fb6f00a3db0 0x7fb6f00a6270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb70810f580 0x7fb70819fbf0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb70806b1b0 0x7fb70819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 --2- 172.21.15.37:0/3946837829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb70806bad0 0x7fb70819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.494+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 >> 172.21.15.37:0/3946837829 conn(0x7fb708077990 msgr2=0x7fb708110b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.495+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 shutdown_connections 2024-04-25T21:26:30.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:30.495+0000 7fb70cf5f700 1 -- 172.21.15.37:0/3946837829 wait complete. 2024-04-25T21:26:30.496 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T21:26:31.182 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:30 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3946837829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:31.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:26:31.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:30 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3946837829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:31.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:30 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3946837829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:26:31.468 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:32.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:31 smithi037 ceph-mon[29195]: pgmap v568: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:32.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:31 smithi177 ceph-mon[33043]: pgmap v568: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:32.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:31 smithi098 ceph-mon[32569]: pgmap v568: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- 172.21.15.37:0/3927438819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 msgr2=0x7f482c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 --2- 172.21.15.37:0/3927438819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c06b630 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f481c0099c0 tx=0x7f481c009cd0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- 172.21.15.37:0/3927438819 shutdown_connections 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 --2- 172.21.15.37:0/3927438819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c06bb70 0x7f482c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 --2- 172.21.15.37:0/3927438819 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c108a50 0x7f482c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 --2- 172.21.15.37:0/3927438819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c06b630 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- 172.21.15.37:0/3927438819 >> 172.21.15.37:0/3927438819 conn(0x7f482c077990 msgr2=0x7f482c077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- 172.21.15.37:0/3927438819 shutdown_connections 2024-04-25T21:26:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- 172.21.15.37:0/3927438819 wait complete. 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 Processor -- start 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.270+0000 7f483349e700 1 -- start start 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c06bb70 0x7f482c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c108a50 0x7f482c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112c40 con 0x7f482c06b1b0 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112ac0 con 0x7f482c06bb70 2024-04-25T21:26:33.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483349e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f482c112dc0 con 0x7f482c108a50 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37316/0 (socket says 172.21.15.37:37316) 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 -- 172.21.15.37:0/2072235442 learned_addr learned my addr 172.21.15.37:0/2072235442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f4830a39700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c06bb70 0x7f482c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f4831a3b700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c108a50 0x7f482c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 -- 172.21.15.37:0/2072235442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c108a50 msgr2=0x7f482c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c108a50 0x7f482c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 -- 172.21.15.37:0/2072235442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c06bb70 msgr2=0x7f482c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c06bb70 0x7f482c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 -- 172.21.15.37:0/2072235442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19e0e0 con 0x7f482c06b1b0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.271+0000 7f483123a700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f482800c9b0 tx=0x7f482800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.272+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828007b00 con 0x7f482c06b1b0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.272+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19e3d0 con 0x7f482c06b1b0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.272+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482800ce50 con 0x7f482c06b1b0 2024-04-25T21:26:33.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.272+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828018660 con 0x7f482c06b1b0 2024-04-25T21:26:33.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.272+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a5c50 con 0x7f482c06b1b0 2024-04-25T21:26:33.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.273+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c069730 con 0x7f482c06b1b0 2024-04-25T21:26:33.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.274+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f482800f450 con 0x7f482c06b1b0 2024-04-25T21:26:33.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.274+0000 7f48227fc700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 0x7f48180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:33.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.275+0000 7f4830a39700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 0x7f48180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:33.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.275+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f48280d0710 con 0x7f482c06b1b0 2024-04-25T21:26:33.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.275+0000 7f4830a39700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 0x7f48180a6270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f481c009990 tx=0x7f481c009650 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:33.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.279+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4828014030 con 0x7f482c06b1b0 2024-04-25T21:26:33.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.491+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f482c19a460 con 0x7f482c06b1b0 2024-04-25T21:26:33.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.492+0000 7f48227fc700 1 -- 172.21.15.37:0/2072235442 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+23042 (secure 0 0 0) 0x7f482c19a460 con 0x7f482c06b1b0 2024-04-25T21:26:33.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T21:26:33.493 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":115,"fsid":"1788dafa-0346-11ef-bc93-c7b262605968","created":"2024-04-25T20:56:30.055661+0000","modified":"2024-04-25T21:26:15.540909+0000","last_up_change":"2024-04-25T21:03:53.105162+0000","last_in_change":"2024-04-25T21:03:32.030880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T21:00:00.177409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T21:04:56.536477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T21:04:59.835175+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a581ae-b05a-4399-8fc9-4fe21b79d2bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6803","nonce":1094372261}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6805","nonce":1094372261}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6809","nonce":1094372261}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":1094372261},{"type":"v1","addr":"172.21.15.37:6807","nonce":1094372261}]},"public_addr":"172.21.15.37:6803/1094372261","cluster_addr":"172.21.15.37:6805/1094372261","heartbeat_back_addr":"172.21.15.37:6809/1094372261","heartbeat_front_addr":"172.21.15.37:6807/1094372261","state":["exists","up"]},{"osd":1,"uuid":"d16bca01-f418-416e-9d6a-57b627ccd028","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6801","nonce":1749616660}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6803","nonce":1749616660}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6807","nonce":1749616660}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":1749616660},{"type":"v1","addr":"172.21.15.98:6805","nonce":1749616660}]},"public_addr":"172.21.15.98:6801/1749616660","cluster_addr":"172.21.15.98:6803/1749616660","heartbeat_back_addr":"172.21.15.98:6807/1749616660","heartbeat_front_addr":"172.21.15.98:6805/1749616660","state":["exists","up"]},{"osd":2,"uuid":"25aa2e48-4132-499e-8958-d7f5b4638000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6801","nonce":5263269}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6803","nonce":5263269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6807","nonce":5263269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":5263269},{"type":"v1","addr":"172.21.15.177:6805","nonce":5263269}]},"public_addr":"172.21.15.177:6801/5263269","cluster_addr":"172.21.15.177:6803/5263269","heartbeat_back_addr":"172.21.15.177:6807/5263269","heartbeat_front_addr":"172.21.15.177:6805/5263269","state":["exists","up"]},{"osd":3,"uuid":"dd7eb40c-16dd-48be-b06b-2278b4ea4333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6811","nonce":3382087416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6813","nonce":3382087416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6817","nonce":3382087416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":3382087416},{"type":"v1","addr":"172.21.15.37:6815","nonce":3382087416}]},"public_addr":"172.21.15.37:6811/3382087416","cluster_addr":"172.21.15.37:6813/3382087416","heartbeat_back_addr":"172.21.15.37:6817/3382087416","heartbeat_front_addr":"172.21.15.37:6815/3382087416","state":["exists","up"]},{"osd":4,"uuid":"363aa3e4-6b35-48ce-a3a1-7e4b21c47651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6809","nonce":2300793190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6811","nonce":2300793190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6815","nonce":2300793190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2300793190},{"type":"v1","addr":"172.21.15.98:6813","nonce":2300793190}]},"public_addr":"172.21.15.98:6809/2300793190","cluster_addr":"172.21.15.98:6811/2300793190","heartbeat_back_addr":"172.21.15.98:6815/2300793190","heartbeat_front_addr":"172.21.15.98:6813/2300793190","state":["exists","up"]},{"osd":5,"uuid":"baed219b-bf19-48da-a89d-13bb3ff8485f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6809","nonce":4266716104}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6811","nonce":4266716104}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6815","nonce":4266716104}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":4266716104},{"type":"v1","addr":"172.21.15.177:6813","nonce":4266716104}]},"public_addr":"172.21.15.177:6809/4266716104","cluster_addr":"172.21.15.177:6811/4266716104","heartbeat_back_addr":"172.21.15.177:6815/4266716104","heartbeat_front_addr":"172.21.15.177:6813/4266716104","state":["exists","up"]},{"osd":6,"uuid":"54c67531-e8c5-4746-bfed-c8c47f4e63de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6819","nonce":3655908250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6821","nonce":3655908250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6825","nonce":3655908250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3655908250},{"type":"v1","addr":"172.21.15.37:6823","nonce":3655908250}]},"public_addr":"172.21.15.37:6819/3655908250","cluster_addr":"172.21.15.37:6821/3655908250","heartbeat_back_addr":"172.21.15.37:6825/3655908250","heartbeat_front_addr":"172.21.15.37:6823/3655908250","state":["exists","up"]},{"osd":7,"uuid":"c7826cf0-a44f-4917-bec2-bcb8aa242bd9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6817","nonce":3844783138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6818","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6819","nonce":3844783138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6823","nonce":3844783138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6820","nonce":3844783138},{"type":"v1","addr":"172.21.15.98:6821","nonce":3844783138}]},"public_addr":"172.21.15.98:6817/3844783138","cluster_addr":"172.21.15.98:6819/3844783138","heartbeat_back_addr":"172.21.15.98:6823/3844783138","heartbeat_front_addr":"172.21.15.98:6821/3844783138","state":["exists","up"]},{"osd":8,"uuid":"2de6cccb-3606-4ff8-9aaa-fda637fade5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6817","nonce":396846600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6819","nonce":396846600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6823","nonce":396846600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":396846600},{"type":"v1","addr":"172.21.15.177:6821","nonce":396846600}]},"public_addr":"172.21.15.177:6817/396846600","cluster_addr":"172.21.15.177:6819/396846600","heartbeat_back_addr":"172.21.15.177:6823/396846600","heartbeat_front_addr":"172.21.15.177:6821/396846600","state":["exists","up"]},{"osd":9,"uuid":"4fa00cfe-c3d2-4741-a451-b8a4c89b6753","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6827","nonce":884722198}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6829","nonce":884722198}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6833","nonce":884722198}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":884722198},{"type":"v1","addr":"172.21.15.37:6831","nonce":884722198}]},"public_addr":"172.21.15.37:6827/884722198","cluster_addr":"172.21.15.37:6829/884722198","heartbeat_back_addr":"172.21.15.37:6833/884722198","heartbeat_front_addr":"172.21.15.37:6831/884722198","state":["exists","up"]},{"osd":10,"uuid":"3397e1ac-4efe-4cd8-8a27-a2f07662d081","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6824","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6825","nonce":592671509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6827","nonce":592671509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6831","nonce":592671509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":592671509},{"type":"v1","addr":"172.21.15.98:6829","nonce":592671509}]},"public_addr":"172.21.15.98:6825/592671509","cluster_addr":"172.21.15.98:6827/592671509","heartbeat_back_addr":"172.21.15.98:6831/592671509","heartbeat_front_addr":"172.21.15.98:6829/592671509","state":["exists","up"]},{"osd":11,"uuid":"52308cf8-3492-4a38-b722-87b7bf3fb5c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6825","nonce":2245259996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6827","nonce":2245259996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6831","nonce":2245259996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":2245259996},{"type":"v1","addr":"172.21.15.177:6829","nonce":2245259996}]},"public_addr":"172.21.15.177:6825/2245259996","cluster_addr":"172.21.15.177:6827/2245259996","heartbeat_back_addr":"172.21.15.177:6831/2245259996","heartbeat_front_addr":"172.21.15.177:6829/2245259996","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:08.395604+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:32.365390+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:59:56.591438+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:23.335461+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:00:48.378350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:13.124948+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:01:39.615684+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:05.162785+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:30.712385+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:02:57.077317+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:23.777752+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T21:03:50.615746+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/1634118111":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/3727603400":"2024-04-26T21:08:28.094244+0000","172.21.15.37:0/234140491":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/2407888079":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6801/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:6800/3898721946":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/4006052172":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1653551922":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1698545333":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/767353771":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/3184535068":"2024-04-26T20:57:34.092925+0000","172.21.15.37:0/1398754476":"2024-04-26T21:08:28.094244+0000","172.21.15.37:6800/1093601384":"2024-04-26T20:57:06.373285+0000","172.21.15.37:0/1828557826":"2024-04-26T20:57:06.373285+0000","172.21.15.37:6801/1093601384":"2024-04-26T20:57:06.373285+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:26:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 msgr2=0x7f48180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:33.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 0x7f48180a6270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f481c009990 tx=0x7f481c009650 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 msgr2=0x7f482c1990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f482800c9b0 tx=0x7f482800ccc0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 shutdown_connections 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f48180a3db0 0x7f48180a6270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f482c108a50 0x7f482c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f482c06b1b0 0x7f482c1990d0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.494+0000 7f483349e700 1 --2- 172.21.15.37:0/2072235442 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f482c06bb70 0x7f482c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.495+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 >> 172.21.15.37:0/2072235442 conn(0x7f482c077990 msgr2=0x7f482c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.495+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 shutdown_connections 2024-04-25T21:26:33.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:33.495+0000 7f483349e700 1 -- 172.21.15.37:0/2072235442 wait complete. 2024-04-25T21:26:34.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:33 smithi037 ceph-mon[29195]: pgmap v569: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:34.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:33 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/2072235442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:26:34.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T21:26:34.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:33 smithi177 ceph-mon[33043]: pgmap v569: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:34.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:33 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/2072235442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:26:34.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:33 smithi098 ceph-mon[32569]: pgmap v569: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:34.417 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:33 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/2072235442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:26:34.488 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:35.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:34 smithi037 ceph-mon[29195]: pgmap v570: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:34 smithi177 ceph-mon[33043]: pgmap v570: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:35.416 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:34 smithi098 ceph-mon[32569]: pgmap v570: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-25T21:26:36.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 -- 172.21.15.37:0/1523284908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4107990 msgr2=0x7f3fa4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/1523284908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4107990 0x7f3fa4107d70 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009990 tx=0x7f3f98009ca0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 -- 172.21.15.37:0/1523284908 shutdown_connections 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/1523284908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4102300 0x7f3fa410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/1523284908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4107990 0x7f3fa4107d70 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/1523284908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4101940 0x7f3fa4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 -- 172.21.15.37:0/1523284908 >> 172.21.15.37:0/1523284908 conn(0x7f3fa40788f0 msgr2=0x7f3fa4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 -- 172.21.15.37:0/1523284908 shutdown_connections 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.231+0000 7f3fab0d1700 1 -- 172.21.15.37:0/1523284908 wait complete. 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 Processor -- start 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 -- start start 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:36.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4102300 0x7f3fa4103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4107990 0x7f3fa4103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa4111bc0 con 0x7f3fa4101940 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa4111a40 con 0x7f3fa4107990 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fab0d1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa4111d40 con 0x7f3fa4102300 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fa8e6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fa8e6d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37324/0 (socket says 172.21.15.37:37324) 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.232+0000 7f3fa8e6d700 1 -- 172.21.15.37:0/346337317 learned_addr learned my addr 172.21.15.37:0/346337317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa3fff700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4102300 0x7f3fa4103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa966e700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4107990 0x7f3fa4103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 -- 172.21.15.37:0/346337317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4102300 msgr2=0x7f3fa4103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4102300 0x7f3fa4103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 -- 172.21.15.37:0/346337317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4107990 msgr2=0x7f3fa4103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4107990 0x7f3fa4103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 -- 172.21.15.37:0/346337317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa41a76f0 con 0x7f3fa4101940 2024-04-25T21:26:36.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa8e6d700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005910 tx=0x7f3f980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98027070 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa41a7980 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98003f20 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.233+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98022c70 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.234+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa41a7e30 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.235+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa4069730 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.236+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f9802a070 con 0x7f3fa4101940 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.236+0000 7f3fa1ffb700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 0x7f3f940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:36.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.237+0000 7f3fa3fff700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 0x7f3f940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.237+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8513+0+0 (secure 0 0 0) 0x7f3f980d1d80 con 0x7f3fa4101940 2024-04-25T21:26:36.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.237+0000 7f3fa3fff700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 0x7f3f940a6270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4078ef0 tx=0x7f3f9000b3c0 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:36.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.240+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f98031070 con 0x7f3fa4101940 2024-04-25T21:26:36.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.495+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3fa41a80c0 con 0x7f3fa4101940 2024-04-25T21:26:36.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.497+0000 7f3fa1ffb700 1 -- 172.21.15.37:0/346337317 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f3f98013070 con 0x7f3fa4101940 2024-04-25T21:26:36.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 msgr2=0x7f3f940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:36.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 0x7f3f940a6270 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4078ef0 tx=0x7f3f9000b3c0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 msgr2=0x7f3fa41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005910 tx=0x7f3f980047b0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 shutdown_connections 2024-04-25T21:26:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f3f940a3db0 0x7f3f940a6270 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3fa4107990 0x7f3fa4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fa4101940 0x7f3fa41034f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 --2- 172.21.15.37:0/346337317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fa4102300 0x7f3fa4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 >> 172.21.15.37:0/346337317 conn(0x7f3fa40788f0 msgr2=0x7f3fa406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.501+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 shutdown_connections 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:36.502+0000 7f3fab0d1700 1 -- 172.21.15.37:0/346337317 wait complete. 2024-04-25T21:26:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T21:26:36.839 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.839 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.839 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-mds[63742]: -- 172.21.15.177:0/4293099091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a5208ba40 con 0x558a5205ec00 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: starting mds.i at 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-mds[63439]: -- 172.21.15.177:0/1413969523 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea720b9a40 con 0x55ea7208d000 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: starting mds.c at 2024-04-25T21:26:36.840 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-mds[63130]: -- 172.21.15.177:0/1950878955 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f78c295a40 con 0x55f78c268800 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: starting mds.f at 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-mds[62821]: -- 172.21.15.177:0/3343649788 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c01dca5a40 con 0x55c01dc78c00 2024-04-25T21:26:36.841 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:26:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: starting mds.l at 2024-04-25T21:26:36.901 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:26:36.496+0000 7f492904c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:26:36.901 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-mds[72194]: -- 172.21.15.37:0/3596947729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560435d49a40 con 0x560435d1c800 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: starting mds.a at 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-mds[71834]: -- 172.21.15.37:0/1497101949 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c39e0d3a40 con 0x55c39e0a7000 2024-04-25T21:26:36.902 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:26:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: starting mds.g at 2024-04-25T21:26:36.916 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.916 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.916 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-mds[62027]: -- 172.21.15.98:0/3481634421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559350ae3a40 con 0x559350ab6800 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: starting mds.k at 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b[62963]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b[62963]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-mds[62968]: -- 172.21.15.98:0/2517520715 <== mon.2 v2:172.21.15.98:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56005b7c5a40 con 0x56005b798c00 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b[62963]: starting mds.b at 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.917 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-mds[62344]: -- 172.21.15.98:0/3631106153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55beb5207a40 con 0x55beb51dac00 2024-04-25T21:26:36.918 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: starting mds.e at 2024-04-25T21:26:36.918 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: ignoring --setuser ceph since I am not root 2024-04-25T21:26:36.918 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: ignoring --setgroup ceph since I am not root 2024-04-25T21:26:36.918 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-mds[62656]: -- 172.21.15.98:0/2759015389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55953e8bfa40 con 0x55953e892800 2024-04-25T21:26:36.918 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:26:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: starting mds.h at 2024-04-25T21:26:37.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: pgmap v571: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T21:26:37.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:37 smithi037 ceph-mon[29195]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T21:26:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:37.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: pgmap v571: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T21:26:37.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:37 smithi177 ceph-mon[33043]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T21:26:37.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: pgmap v571: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/346337317' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: osdmap e116: 12 total, 12 up, 12 in 2024-04-25T21:26:37.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:37 smithi098 ceph-mon[32569]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6838/1621201115,v1:172.21.15.37:6839/1621201115] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.37:6840/4221813810,v1:172.21.15.37:6841/4221813810] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6833/1459395465,v1:172.21.15.98:6835/1459395465] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6832/1277041428,v1:172.21.15.98:6834/1277041428] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6832/3101651919,v1:172.21.15.177:6833/3101651919] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6834/143983362,v1:172.21.15.177:6835/143983362] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6836/648176459,v1:172.21.15.177:6837/648176459] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6836/3128053653,v1:172.21.15.98:6838/3128053653] up:boot 2024-04-25T21:26:38.638 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.177:6838/1164620880,v1:172.21.15.177:6839/1164620880] up:boot 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: mds.? [v2:172.21.15.98:6837/2005346678,v1:172.21.15.98:6839/2005346678] up:boot 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:26:38.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:26:38.640 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:38 smithi037 ceph-mon[29195]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6838/1621201115,v1:172.21.15.37:6839/1621201115] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.37:6840/4221813810,v1:172.21.15.37:6841/4221813810] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6833/1459395465,v1:172.21.15.98:6835/1459395465] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6832/1277041428,v1:172.21.15.98:6834/1277041428] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6832/3101651919,v1:172.21.15.177:6833/3101651919] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6834/143983362,v1:172.21.15.177:6835/143983362] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6836/648176459,v1:172.21.15.177:6837/648176459] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6836/3128053653,v1:172.21.15.98:6838/3128053653] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.177:6838/1164620880,v1:172.21.15.177:6839/1164620880] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: mds.? [v2:172.21.15.98:6837/2005346678,v1:172.21.15.98:6839/2005346678] up:boot 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T21:26:38.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:26:38.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:38 smithi177 ceph-mon[33043]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:26:38.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6838/1621201115,v1:172.21.15.37:6839/1621201115] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.37:6840/4221813810,v1:172.21.15.37:6841/4221813810] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6833/1459395465,v1:172.21.15.98:6835/1459395465] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6832/1277041428,v1:172.21.15.98:6834/1277041428] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6832/3101651919,v1:172.21.15.177:6833/3101651919] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6834/143983362,v1:172.21.15.177:6835/143983362] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6836/648176459,v1:172.21.15.177:6837/648176459] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6836/3128053653,v1:172.21.15.98:6838/3128053653] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.177:6838/1164620880,v1:172.21.15.177:6839/1164620880] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: mds.? [v2:172.21.15.98:6837/2005346678,v1:172.21.15.98:6839/2005346678] up:boot 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:26:38.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:26:38.918 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:38 smithi098 ceph-mon[32569]: from='mgr.24830 172.21.15.37:0/2594263961' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T21:26:39.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 -- 172.21.15.37:0/2337883298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 msgr2=0x7f6834108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:39.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 --2- 172.21.15.37:0/2337883298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f6834108e40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6824009990 tx=0x7f6824009ca0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 -- 172.21.15.37:0/2337883298 shutdown_connections 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 --2- 172.21.15.37:0/2337883298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 0x7f683410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 --2- 172.21.15.37:0/2337883298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f6834108e40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 --2- 172.21.15.37:0/2337883298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 0x7f683406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 -- 172.21.15.37:0/2337883298 >> 172.21.15.37:0/2337883298 conn(0x7f68340789e0 msgr2=0x7f6834078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 -- 172.21.15.37:0/2337883298 shutdown_connections 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 -- 172.21.15.37:0/2337883298 wait complete. 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.217+0000 7f683959b700 1 Processor -- start 2024-04-25T21:26:39.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 -- start start 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 0x7f68341991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 0x7f68341996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112c40 con 0x7f6834108a60 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112ac0 con 0x7f683406b1f0 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f683959b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112dc0 con 0x7f683406bbb0 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f68337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f68337fe700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45040/0 (socket says 172.21.15.37:45040) 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f68337fe700 1 -- 172.21.15.37:0/1007619628 learned_addr learned my addr 172.21.15.37:0/1007619628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f68327fc700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 0x7f68341996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.218+0000 7f6832ffd700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 0x7f68341991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 -- 172.21.15.37:0/1007619628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 msgr2=0x7f68341996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 0x7f68341996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 -- 172.21.15.37:0/1007619628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 msgr2=0x7f68341991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 0x7f68341991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 -- 172.21.15.37:0/1007619628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683419e1b0 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f68337fe700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f682c007d70 tx=0x7f682c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c016020 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c00ac90 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683419e4a0 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c0115e0 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.219+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a5d20 con 0x7f6834108a60 2024-04-25T21:26:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.220+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683410b060 con 0x7f6834108a60 2024-04-25T21:26:39.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.222+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f682c019020 con 0x7f6834108a60 2024-04-25T21:26:39.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.222+0000 7f681bfff700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 0x7f681c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.222+0000 7f6832ffd700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 0x7f681c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.222+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8899+0+0 (secure 0 0 0) 0x7f682c0d0be0 con 0x7f6834108a60 2024-04-25T21:26:39.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.223+0000 7f6832ffd700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 0x7f681c0a6270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68240098e0 tx=0x7f6824005960 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:39.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.226+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f682c020020 con 0x7f6834108a60 2024-04-25T21:26:39.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.479+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f683419a500 con 0x7f6834108a60 2024-04-25T21:26:39.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.507+0000 7f681bfff700 1 -- 172.21.15.37:0/1007619628 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f682c01ae40 con 0x7f6834108a60 2024-04-25T21:26:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 msgr2=0x7f681c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 0x7f681c0a6270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f68240098e0 tx=0x7f6824005960 comp rx=0 tx=0).stop 2024-04-25T21:26:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 msgr2=0x7f683419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f682c007d70 tx=0x7f682c00a340 comp rx=0 tx=0).stop 2024-04-25T21:26:39.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 shutdown_connections 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f681c0a3db0 0x7f681c0a6270 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834108a60 0x7f683419da70 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f683406b1f0 0x7f68341991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 --2- 172.21.15.37:0/1007619628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f683406bbb0 0x7f68341996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 >> 172.21.15.37:0/1007619628 conn(0x7f68340789e0 msgr2=0x7f68340ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 shutdown_connections 2024-04-25T21:26:39.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:39.509+0000 7f683959b700 1 -- 172.21.15.37:0/1007619628 wait complete. 2024-04-25T21:26:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:39 smithi037 ceph-mon[29195]: pgmap v573: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-25T21:26:39.734 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:39 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:26:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:39 smithi177 ceph-mon[33043]: pgmap v573: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-25T21:26:39.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:39 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:26:39.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:39 smithi098 ceph-mon[32569]: pgmap v573: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-25T21:26:39.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:39 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:26:40.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T21:26:40.471 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:40 smithi177 ceph-mon[33043]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:26:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:40 smithi177 ceph-mon[33043]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:26:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:40 smithi177 ceph-mon[33043]: Cluster is now healthy 2024-04-25T21:26:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:40 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:26:40.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:40 smithi177 ceph-mon[33043]: fsmap 12 up:standby 2024-04-25T21:26:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:40 smithi037 ceph-mon[29195]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:26:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:40 smithi037 ceph-mon[29195]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:26:40.851 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:40 smithi037 ceph-mon[29195]: Cluster is now healthy 2024-04-25T21:26:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:40 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:26:40.852 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:40 smithi037 ceph-mon[29195]: fsmap 12 up:standby 2024-04-25T21:26:40.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:40 smithi098 ceph-mon[32569]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:26:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:40 smithi098 ceph-mon[32569]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:26:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:40 smithi098 ceph-mon[32569]: Cluster is now healthy 2024-04-25T21:26:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:40 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/1007619628' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:26:40.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:40 smithi098 ceph-mon[32569]: fsmap 12 up:standby 2024-04-25T21:26:41.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:41 smithi037 ceph-mon[29195]: pgmap v574: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:41.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:41 smithi177 ceph-mon[33043]: pgmap v574: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:41.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:41 smithi098 ceph-mon[32569]: pgmap v574: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:42.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.212+0000 7f4603a0d700 1 -- 172.21.15.37:0/3274073051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc101400 msgr2=0x7f45fc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:42.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.212+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3274073051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc101400 0x7f45fc101880 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009990 tx=0x7f45ec009ca0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 -- 172.21.15.37:0/3274073051 shutdown_connections 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3274073051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101dc0 0x7f45fc10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3274073051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc107b20 0x7f45fc107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3274073051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc101400 0x7f45fc101880 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 -- 172.21.15.37:0/3274073051 >> 172.21.15.37:0/3274073051 conn(0x7f45fc076c60 msgr2=0x7f45fc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 -- 172.21.15.37:0/3274073051 shutdown_connections 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 -- 172.21.15.37:0/3274073051 wait complete. 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 Processor -- start 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.213+0000 7f4603a0d700 1 -- start start 2024-04-25T21:26:42.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101400 0x7f45fc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc101dc0 0x7f45fc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc112cb0 con 0x7f45fc107b20 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc112b30 con 0x7f45fc101400 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4603a0d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc112e30 con 0x7f45fc101dc0 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45058/0 (socket says 172.21.15.37:45058) 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 -- 172.21.15.37:0/3368347797 learned_addr learned my addr 172.21.15.37:0/3368347797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f46017a9700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101400 0x7f45fc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4600fa8700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc101dc0 0x7f45fc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 -- 172.21.15.37:0/3368347797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc101dc0 msgr2=0x7f45fc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:42.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc101dc0 0x7f45fc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 -- 172.21.15.37:0/3368347797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101400 msgr2=0x7f45fc199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:42.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101400 0x7f45fc199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 -- 172.21.15.37:0/3368347797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc19e110 con 0x7f45fc107b20 2024-04-25T21:26:42.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f4601faa700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f45f400ada0 tx=0x7f45f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:42.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.214+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4017020 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.215+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f400ce30 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.215+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc19e400 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.215+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4005350 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.215+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1a5c80 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.216+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc069730 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.217+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45f400c980 con 0x7f45fc107b20 2024-04-25T21:26:42.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.217+0000 7f45f27fc700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 0x7f45e80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:42.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.218+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8899+0+0 (secure 0 0 0) 0x7f45f40d0670 con 0x7f45fc107b20 2024-04-25T21:26:42.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.218+0000 7f46017a9700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 0x7f45e80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:42.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.218+0000 7f46017a9700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 0x7f45e80a62f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0781e0 tx=0x7f45f800b410 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:42.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.221+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45f4021020 con 0x7f45fc107b20 2024-04-25T21:26:42.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.429+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45fc02df80 con 0x7f45fc107b20 2024-04-25T21:26:42.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.526+0000 7f45f27fc700 1 -- 172.21.15.37:0/3368347797 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f45f401a310 con 0x7f45fc107b20 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 msgr2=0x7f45e80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 0x7f45e80a62f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f45fc0781e0 tx=0x7f45f800b410 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 msgr2=0x7f45fc19d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f45f400ada0 tx=0x7f45f400c330 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 shutdown_connections 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f45e80a3e30 0x7f45e80a62f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45fc107b20 0x7f45fc19d9d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45fc101400 0x7f45fc199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 --2- 172.21.15.37:0/3368347797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45fc101dc0 0x7f45fc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 >> 172.21.15.37:0/3368347797 conn(0x7f45fc076c60 msgr2=0x7f45fc100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 shutdown_connections 2024-04-25T21:26:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:42.528+0000 7f4603a0d700 1 -- 172.21.15.37:0/3368347797 wait complete. 2024-04-25T21:26:42.528 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T21:26:42.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:42 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:42.848 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:42 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:42.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:42 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:43.188 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 1788dafa-0346-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T21:26:43.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:26:43.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:43 smithi037 ceph-mon[29195]: pgmap v575: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:43.827 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:43 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:43.828 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:43 smithi037 ceph-mon[29195]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T21:26:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:43 smithi177 ceph-mon[33043]: pgmap v575: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:43 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:43.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:43 smithi177 ceph-mon[33043]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T21:26:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:43 smithi098 ceph-mon[32569]: pgmap v575: 97 pgs: 97 active+clean; 5.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:26:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:43 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3368347797' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:43.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:43 smithi098 ceph-mon[32569]: osdmap e117: 12 total, 12 up, 12 in 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.171+0000 7f089e605700 1 -- 172.21.15.37:0/3627599300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 msgr2=0x7f0898101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.171+0000 7f089e605700 1 --2- 172.21.15.37:0/3627599300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898101dc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f088c009990 tx=0x7f088c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.171+0000 7f089e605700 1 -- 172.21.15.37:0/3627599300 shutdown_connections 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.171+0000 7f089e605700 1 --2- 172.21.15.37:0/3627599300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898102300 0x7f089810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 --2- 172.21.15.37:0/3627599300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898107990 0x7f0898107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 --2- 172.21.15.37:0/3627599300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898101dc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 -- 172.21.15.37:0/3627599300 >> 172.21.15.37:0/3627599300 conn(0x7f08980788f0 msgr2=0x7f0898078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 -- 172.21.15.37:0/3627599300 shutdown_connections 2024-04-25T21:26:45.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 -- 172.21.15.37:0/3627599300 wait complete. 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 Processor -- start 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.172+0000 7f089e605700 1 -- start start 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898102300 0x7f08981986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898107990 0x7f089819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111bd0 con 0x7f0898101940 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111a50 con 0x7f0898107990 2024-04-25T21:26:45.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089e605700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111d50 con 0x7f0898102300 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45080/0 (socket says 172.21.15.37:45080) 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 -- 172.21.15.37:0/3396760240 learned_addr learned my addr 172.21.15.37:0/3396760240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f08977fe700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898102300 0x7f08981986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f089cba2700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898107990 0x7f089819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 -- 172.21.15.37:0/3396760240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898102300 msgr2=0x7f08981986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898102300 0x7f08981986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 -- 172.21.15.37:0/3396760240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898107990 msgr2=0x7f089819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898107990 0x7f089819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.173+0000 7f0897fff700 1 -- 172.21.15.37:0/3396760240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089819d170 con 0x7f0898101940 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f0897fff700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f088800b530 tx=0x7f088800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888015020 con 0x7f0898101940 2024-04-25T21:26:45.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0888011ce0 con 0x7f0898101940 2024-04-25T21:26:45.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089819d460 con 0x7f0898101940 2024-04-25T21:26:45.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888010d20 con 0x7f0898101940 2024-04-25T21:26:45.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.174+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08981a4ce0 con 0x7f0898101940 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.175+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0898109eb0 con 0x7f0898101940 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.176+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0888018020 con 0x7f0898101940 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.177+0000 7f08957fa700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 0x7f08800a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.177+0000 7f08977fe700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 0x7f08800a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.177+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8403+0+0 (secure 0 0 0) 0x7f08880cfeb0 con 0x7f0898101940 2024-04-25T21:26:45.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.177+0000 7f08977fe700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 0x7f08800a62f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c01a040 comp rx=0 tx=0).ready entity=mgr.24830 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:26:45.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.181+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f088801f020 con 0x7f0898101940 2024-04-25T21:26:45.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.393+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0898069730 con 0x7f0898101940 2024-04-25T21:26:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.529+0000 7f08957fa700 1 -- 172.21.15.37:0/3396760240 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7f088809c6d0 con 0x7f0898101940 2024-04-25T21:26:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 msgr2=0x7f08800a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 0x7f08800a62f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f088c006010 tx=0x7f088c01a040 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 msgr2=0x7f0898198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f088800b530 tx=0x7f088800b8f0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 shutdown_connections 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:6800/4256525286,v1:172.21.15.37:6801/4256525286] conn(0x7f08800a3e30 0x7f08800a62f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0898107990 0x7f089819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0898101940 0x7f0898198160 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 --2- 172.21.15.37:0/3396760240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0898102300 0x7f08981986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.532+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 >> 172.21.15.37:0/3396760240 conn(0x7f08980788f0 msgr2=0x7f08980ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.533+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 shutdown_connections 2024-04-25T21:26:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T21:26:45.533+0000 7f089e605700 1 -- 172.21.15.37:0/3396760240 wait complete. 2024-04-25T21:26:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T21:26:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:45 smithi177 ceph-mon[33043]: pgmap v577: 65 pgs: 65 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:26:45.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:45 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:45.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:45 smithi037 ceph-mon[29195]: pgmap v577: 65 pgs: 65 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:26:45.858 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:45 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:45 smithi098 ceph-mon[32569]: pgmap v577: 65 pgs: 65 active+clean; 577 KiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:26:45.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:45 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:26:46.214 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:26:46.243 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:26:46.275 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:26:46.310 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T21:26:46.310 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:26:46.326 DEBUG:teuthology.orchestra.run.smithi098:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:26:46.342 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:26:46.369 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T21:26:46.369 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T21:26:46.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a 2024-04-25T21:26:46.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 systemd[1]: Stopping Ceph mon.a for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:46.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:46 smithi177 ceph-mon[33043]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:46.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:46 smithi177 ceph-mon[33043]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T21:26:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:46 smithi098 ceph-mon[32569]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:46.916 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:46 smithi098 ceph-mon[32569]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T21:26:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 ceph-mon[29195]: from='client.? 172.21.15.37:0/3396760240' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:26:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 ceph-mon[29195]: osdmap e118: 12 total, 12 up, 12 in 2024-04-25T21:26:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:26:46.679+0000 7f492e857700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:26:46.924 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a[29191]: 2024-04-25T21:26:46.679+0000 7f492e857700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T21:26:47.222 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:46 smithi037 podman[109059]: 2024-04-25 21:26:46.925475999 +0000 UTC m=+0.361307065 container died 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:26:47.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:47 smithi037 podman[109059]: 2024-04-25 21:26:47.31375451 +0000 UTC m=+0.749585571 container cleanup 08c86468669fea9e12341c6fd08fd12480f8d1c2ac5f9b7e36076adf95c45f3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T21:26:47.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 25 21:26:47 smithi037 bash[109059]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-a 2024-04-25T21:26:47.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T21:26:47.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:47.852 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T21:26:47.852 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T21:26:47.853 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.b 2024-04-25T21:26:48.166 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:47 smithi098 systemd[1]: Stopping Ceph mon.b for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:48.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b[32565]: 2024-04-25T21:26:48.230+0000 7f664c8b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:26:48.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b[32565]: 2024-04-25T21:26:48.230+0000 7f664c8b7700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:26:48.616 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:48 smithi098 podman[69965]: 2024-04-25 21:26:48.303020855 +0000 UTC m=+0.254732983 container died ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T21:26:48.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:48 smithi098 podman[69965]: 2024-04-25 21:26:48.716457282 +0000 UTC m=+0.668169425 container cleanup ff3fce8ce874aed86f3f2bf3d3f3473be0c0add8f3546aba0de30d08d2fa7e75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:26:48.917 INFO:journalctl@ceph.mon.b.smithi098.stdout:Apr 25 21:26:48 smithi098 bash[69965]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-b 2024-04-25T21:26:49.175 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T21:26:49.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:49.222 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T21:26:49.222 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T21:26:49.222 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.c 2024-04-25T21:26:49.546 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:49 smithi177 systemd[1]: Stopping Ceph mon.c for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c[33039]: 2024-04-25T21:26:49.546+0000 7f05df555700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:26:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c[33039]: 2024-04-25T21:26:49.546+0000 7f05df555700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:26:49.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:49 smithi177 podman[71501]: 2024-04-25 21:26:49.719988517 +0000 UTC m=+0.296538181 container died 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:26:50.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:50 smithi177 podman[71501]: 2024-04-25 21:26:50.147198653 +0000 UTC m=+0.723748318 container cleanup 662780321a8adb1f7c71d1bafc4921f95c7afd4a580ae45f49f28cee3665168a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:26:50.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Apr 25 21:26:50 smithi177 bash[71501]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mon-c 2024-04-25T21:26:50.597 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T21:26:50.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:50.642 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T21:26:50.642 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T21:26:50.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x 2024-04-25T21:26:50.959 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:26:50 smithi037 systemd[1]: Stopping Ceph mgr.x for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:51.234 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:26:50 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x[29424]: 2024-04-25T21:26:50.959+0000 7f6809f69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:26:51.234 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:26:50 smithi037 podman[109180]: 2024-04-25 21:26:50.990920739 +0000 UTC m=+0.150733342 container died 4401a6809afe8ca157d868521af846cf655686da706102451ad421f3994c4cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:26:51.734 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:26:51 smithi037 podman[109180]: 2024-04-25 21:26:51.469845608 +0000 UTC m=+0.629658244 container cleanup 4401a6809afe8ca157d868521af846cf655686da706102451ad421f3994c4cfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T21:26:51.734 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 25 21:26:51 smithi037 bash[109180]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-x 2024-04-25T21:26:51.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T21:26:51.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:51.938 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T21:26:51.938 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T21:26:51.938 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.y 2024-04-25T21:26:52.274 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:26:51 smithi098 systemd[1]: Stopping Ceph mgr.y for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:52.572 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:26:52 smithi098 podman[70087]: 2024-04-25 21:26:52.275815818 +0000 UTC m=+0.141327004 container died 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:26:52.915 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:26:52 smithi098 podman[70087]: 2024-04-25 21:26:52.673245798 +0000 UTC m=+0.538756993 container cleanup 6ccdb20392c90fb90ff9a6a36cb71d33829c88be1e570f5fb90a4a56b79e4b05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T21:26:52.915 INFO:journalctl@ceph.mgr.y.smithi098.stdout:Apr 25 21:26:52 smithi098 bash[70087]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-y 2024-04-25T21:26:53.089 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T21:26:53.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:53.122 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T21:26:53.122 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T21:26:53.122 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.z 2024-04-25T21:26:53.465 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:26:53 smithi177 systemd[1]: Stopping Ceph mgr.z for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:53.769 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:26:53 smithi177 podman[71633]: 2024-04-25 21:26:53.465897845 +0000 UTC m=+0.155787439 container died 68329c5653739c94bec4ac6275101669ec51c235a0c8d72a9756a82ba9dd3317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:26:54.089 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:26:53 smithi177 podman[71633]: 2024-04-25 21:26:53.860875478 +0000 UTC m=+0.550765131 container cleanup 68329c5653739c94bec4ac6275101669ec51c235a0c8d72a9756a82ba9dd3317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T21:26:54.089 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Apr 25 21:26:53 smithi177 bash[71633]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mgr-z 2024-04-25T21:26:54.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T21:26:54.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:26:54.273 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T21:26:54.273 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T21:26:54.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.0 2024-04-25T21:26:54.581 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:26:54 smithi037 systemd[1]: Stopping Ceph osd.0 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:26:54.984 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:26:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0[43821]: 2024-04-25T21:26:54.582+0000 7efdee9fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:26:54.984 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:26:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0[43821]: 2024-04-25T21:26:54.582+0000 7efdee9fe700 -1 osd.0 118 *** Got signal Terminated *** 2024-04-25T21:26:54.984 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:26:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0[43821]: 2024-04-25T21:26:54.582+0000 7efdee9fe700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:00.234 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:26:59 smithi037 podman[109300]: 2024-04-25 21:26:59.963173035 +0000 UTC m=+5.497258453 container died c95e96414b6b354daa87ec71aeb3b59e3edd14edc39eeeed98de14538da95282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:27:00.718 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:00 smithi037 podman[109300]: 2024-04-25 21:27:00.467547619 +0000 UTC m=+6.001633070 container cleanup c95e96414b6b354daa87ec71aeb3b59e3edd14edc39eeeed98de14538da95282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T21:27:00.719 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:00 smithi037 bash[109300]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0 2024-04-25T21:27:00.984 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:00 smithi037 podman[109317]: 2024-04-25 21:27:00.719230715 +0000 UTC m=+0.755930924 container remove c95e96414b6b354daa87ec71aeb3b59e3edd14edc39eeeed98de14538da95282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T21:27:01.331 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:00.95056775 +0000 UTC m=+0.018628337 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:01.331 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:01.062081379 +0000 UTC m=+0.130141952 container create 1c70c63dece8e5ce78e8259e203d1678a375857af8e9eace9041826312b609f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T21:27:01.637 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:01.331735333 +0000 UTC m=+0.399795913 container init 1c70c63dece8e5ce78e8259e203d1678a375857af8e9eace9041826312b609f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:01.637 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:01.337142256 +0000 UTC m=+0.405202836 container start 1c70c63dece8e5ce78e8259e203d1678a375857af8e9eace9041826312b609f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T21:27:01.637 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:01.448626575 +0000 UTC m=+0.516687163 container attach 1c70c63dece8e5ce78e8259e203d1678a375857af8e9eace9041826312b609f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T21:27:01.942 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 25 21:27:01 smithi037 podman[109413]: 2024-04-25 21:27:01.638168094 +0000 UTC m=+0.706228675 container died 1c70c63dece8e5ce78e8259e203d1678a375857af8e9eace9041826312b609f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:02.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T21:27:02.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:02.388 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T21:27:02.388 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T21:27:02.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.1 2024-04-25T21:27:02.667 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:02 smithi098 systemd[1]: Stopping Ceph osd.1 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:03.166 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1[37171]: 2024-04-25T21:27:02.710+0000 7fd136a58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:03.167 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1[37171]: 2024-04-25T21:27:02.710+0000 7fd136a58700 -1 osd.1 118 *** Got signal Terminated *** 2024-04-25T21:27:03.167 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1[37171]: 2024-04-25T21:27:02.710+0000 7fd136a58700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:08.570 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:08 smithi098 podman[70212]: 2024-04-25 21:27:08.26630026 +0000 UTC m=+5.684957374 container died ee0e2975008072a96d0eca88b8fca24cc33143a214d8cb062edd04bf8d1841da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:27:08.916 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:08 smithi098 podman[70212]: 2024-04-25 21:27:08.678992317 +0000 UTC m=+6.097649428 container cleanup ee0e2975008072a96d0eca88b8fca24cc33143a214d8cb062edd04bf8d1841da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T21:27:08.917 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:08 smithi098 bash[70212]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1 2024-04-25T21:27:09.291 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:08 smithi098 podman[70230]: 2024-04-25 21:27:08.961352974 +0000 UTC m=+0.689203347 container remove ee0e2975008072a96d0eca88b8fca24cc33143a214d8cb062edd04bf8d1841da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T21:27:09.551 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.195388432 +0000 UTC m=+0.018219091 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:09.551 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.316202909 +0000 UTC m=+0.139033602 container create 6c1234305c1474696b759a3d221239a0fedaacc8ceaffe006b43e956d2036681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:27:09.551 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.552501315 +0000 UTC m=+0.375331966 container init 6c1234305c1474696b759a3d221239a0fedaacc8ceaffe006b43e956d2036681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T21:27:09.842 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.555650407 +0000 UTC m=+0.378481058 container start 6c1234305c1474696b759a3d221239a0fedaacc8ceaffe006b43e956d2036681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:27:09.842 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.677577035 +0000 UTC m=+0.500407694 container attach 6c1234305c1474696b759a3d221239a0fedaacc8ceaffe006b43e956d2036681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T21:27:10.150 INFO:journalctl@ceph.osd.1.smithi098.stdout:Apr 25 21:27:09 smithi098 podman[70323]: 2024-04-25 21:27:09.842761106 +0000 UTC m=+0.665591757 container died 6c1234305c1474696b759a3d221239a0fedaacc8ceaffe006b43e956d2036681 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T21:27:10.472 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T21:27:10.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:10.510 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T21:27:10.510 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T21:27:10.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.2 2024-04-25T21:27:10.839 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:10 smithi177 systemd[1]: Stopping Ceph osd.2 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:11.339 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2[38222]: 2024-04-25T21:27:10.839+0000 7fc14aad6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:11.339 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2[38222]: 2024-04-25T21:27:10.839+0000 7fc14aad6700 -1 osd.2 118 *** Got signal Terminated *** 2024-04-25T21:27:11.339 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2[38222]: 2024-04-25T21:27:10.839+0000 7fc14aad6700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:16.431 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:16 smithi177 podman[71759]: 2024-04-25 21:27:16.180726945 +0000 UTC m=+5.477725187 container died 13e2bbc1ba6b5dcd991b3be4376780549a0f5c2d10f91dc2dbd502fadca44985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T21:27:16.732 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:16 smithi177 podman[71759]: 2024-04-25 21:27:16.523312971 +0000 UTC m=+5.820311204 container cleanup 13e2bbc1ba6b5dcd991b3be4376780549a0f5c2d10f91dc2dbd502fadca44985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:27:16.732 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:16 smithi177 bash[71759]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2 2024-04-25T21:27:17.089 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:16 smithi177 podman[71777]: 2024-04-25 21:27:16.733422174 +0000 UTC m=+0.542334474 container remove 13e2bbc1ba6b5dcd991b3be4376780549a0f5c2d10f91dc2dbd502fadca44985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:27:17.452 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.040168495 +0000 UTC m=+0.018186268 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:17.452 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.151070905 +0000 UTC m=+0.129088716 container create 338b6cee20263190bb9a511f246e4bf0b9d9a0ae9bcf977feb4f7095146018b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T21:27:17.452 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.370434241 +0000 UTC m=+0.348452012 container init 338b6cee20263190bb9a511f246e4bf0b9d9a0ae9bcf977feb4f7095146018b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:17.453 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.373771688 +0000 UTC m=+0.351789451 container start 338b6cee20263190bb9a511f246e4bf0b9d9a0ae9bcf977feb4f7095146018b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:17.839 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.453665834 +0000 UTC m=+0.431683624 container attach 338b6cee20263190bb9a511f246e4bf0b9d9a0ae9bcf977feb4f7095146018b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:27:17.839 INFO:journalctl@ceph.osd.2.smithi177.stdout:Apr 25 21:27:17 smithi177 podman[71869]: 2024-04-25 21:27:17.665479419 +0000 UTC m=+0.643497193 container died 338b6cee20263190bb9a511f246e4bf0b9d9a0ae9bcf977feb4f7095146018b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:27:18.302 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T21:27:18.339 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:18.339 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T21:27:18.339 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T21:27:18.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.3 2024-04-25T21:27:18.664 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:18 smithi037 systemd[1]: Stopping Ceph osd.3 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:18.984 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:18 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3[50762]: 2024-04-25T21:27:18.663+0000 7f827c120700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:18.984 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:18 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3[50762]: 2024-04-25T21:27:18.663+0000 7f827c120700 -1 osd.3 118 *** Got signal Terminated *** 2024-04-25T21:27:18.984 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:18 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3[50762]: 2024-04-25T21:27:18.663+0000 7f827c120700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:21.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:21 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:21.640+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:22.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:21.956+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:22.416 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:22 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:22.055+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:22.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:22 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:22.599+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:23.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:22.928+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:23.416 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:23 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:23.081+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:23.607 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:23.292+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:23.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:23.386+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:23.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:23 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:23.608+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:24.312 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:24 smithi037 podman[109536]: 2024-04-25 21:27:24.026763744 +0000 UTC m=+5.482745838 container died 2532cb3c42d4d84dc916e5b23a1b2a1905948344e54e8b68a848187f52fc3f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:24.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:23.952+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:24.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:24 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:24.087+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:24.621 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:24.312+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:24.621 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:24 smithi037 podman[109536]: 2024-04-25 21:27:24.414559692 +0000 UTC m=+5.870541764 container cleanup 2532cb3c42d4d84dc916e5b23a1b2a1905948344e54e8b68a848187f52fc3f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-25T21:27:24.621 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:24 smithi037 bash[109536]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3 2024-04-25T21:27:24.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:24.393+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:24.984 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:24 smithi037 podman[109554]: 2024-04-25 21:27:24.674497141 +0000 UTC m=+0.642281243 container remove 2532cb3c42d4d84dc916e5b23a1b2a1905948344e54e8b68a848187f52fc3f35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T21:27:24.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:24 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:24.620+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:25.294 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:24 smithi037 podman[109641]: 2024-04-25 21:27:24.898908779 +0000 UTC m=+0.018472813 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:25.294 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:25 smithi037 podman[109641]: 2024-04-25 21:27:25.025258512 +0000 UTC m=+0.144822520 container create d48a4d2078987604ea8dfb171be6cec86e8c15f43c53adf693fb36217c1f79da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T21:27:25.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:24.949+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:25.416 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:25 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:25.107+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:25.597 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:25 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:25.296+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:25.598 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:25 smithi037 podman[109641]: 2024-04-25 21:27:25.295063911 +0000 UTC m=+0.414627931 container init d48a4d2078987604ea8dfb171be6cec86e8c15f43c53adf693fb36217c1f79da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:27:25.598 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:25 smithi037 podman[109641]: 2024-04-25 21:27:25.298726563 +0000 UTC m=+0.418290572 container start d48a4d2078987604ea8dfb171be6cec86e8c15f43c53adf693fb36217c1f79da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:27:25.598 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:25 smithi037 podman[109641]: 2024-04-25 21:27:25.394718195 +0000 UTC m=+0.514282210 container attach d48a4d2078987604ea8dfb171be6cec86e8c15f43c53adf693fb36217c1f79da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:25.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:25.435+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:25.888 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 25 21:27:25 smithi037 podman[109641]: 2024-04-25 21:27:25.599155252 +0000 UTC m=+0.718719268 container died d48a4d2078987604ea8dfb171be6cec86e8c15f43c53adf693fb36217c1f79da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T21:27:25.889 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:25 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:25.611+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:26.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T21:27:26.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:26.249 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T21:27:26.250 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T21:27:26.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.4 2024-04-25T21:27:26.320 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:25.986+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:26.416 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:26 smithi098 systemd[1]: Stopping Ceph osd.4 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:26.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:26.150+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:26.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:26.265+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:26.589 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:26.442+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:26.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:26.320+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:26.917 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:27:26.570+0000 7f4a87b5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:26.917 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:27:26.570+0000 7f4a87b5f700 -1 osd.4 118 *** Got signal Terminated *** 2024-04-25T21:27:26.917 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:26 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:27:26.570+0000 7f4a87b5f700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:26.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:26 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:26.608+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:27.337 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:27.035+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:27.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:27.337+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:27.589 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:27.457+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:27.595 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:27.263+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:27.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:27 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:27.191+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:27.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:27 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:27.595+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:28.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:28 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:28.024+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:28.569 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:28.272+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:28.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:28 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:28.175+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:28.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:28 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:28.381+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:28.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:28 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:28.426+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:28.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:28 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:28.426+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:28.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:28 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:28.570+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:29.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:29.073+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:29.490 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:29 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:29.205+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:29.491 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:29 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:29.205+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:29.548 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:29.287+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:29.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:29.457+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:29.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:29.457+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:29.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:29.428+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:29.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:29 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:29.428+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:29.916 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:29 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:27:29.689+0000 7f4a84153700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:04.532397+0000 front 2024-04-25T21:27:04.532370+0000 (oldest deadline 2024-04-25T21:27:29.232525+0000) 2024-04-25T21:27:29.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:29 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:29.490+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:29.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:29 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:29.549+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:30.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:30 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:30.088+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:30.468 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:30 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:30.174+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:30.468 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:30 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:30.174+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:30.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:30.534+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:30.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:30.294+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:30.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:30 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:30.503+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:30.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:30 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:30.503+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:30.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:30 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:30.415+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:30.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:30 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:30.415+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:30.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:30 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:30.469+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:30.916 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:30 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4[43852]: 2024-04-25T21:27:30.688+0000 7f4a84153700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:04.532397+0000 front 2024-04-25T21:27:04.532370+0000 (oldest deadline 2024-04-25T21:27:29.232525+0000) 2024-04-25T21:27:31.390 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:31 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:31.131+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:31.416 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:31 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:31.132+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:31.416 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:31 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:31.132+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:31.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:31 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:31.490+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:31.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:31 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:31.311+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:31.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:31 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:31.391+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:31.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:31 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:31.391+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:31.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:31 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:31.463+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:31.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:31 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:31.463+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:31.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:31 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:31.516+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:32.330 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:32 smithi098 podman[70441]: 2024-04-25 21:27:32.037849958 +0000 UTC m=+5.590075764 container died 8014a000145e1016590d92cfda62ba3ec87bf327420c3711e83500e6c38bee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:32.330 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:32 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:32.116+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:32.330 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:32 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:32.116+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:32.385 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:32.134+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:32.666 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:32 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:32.496+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:32.667 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:32 smithi098 podman[70441]: 2024-04-25 21:27:32.430555336 +0000 UTC m=+5.982781142 container cleanup 8014a000145e1016590d92cfda62ba3ec87bf327420c3711e83500e6c38bee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T21:27:32.667 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:32 smithi098 bash[70441]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4 2024-04-25T21:27:32.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:32.267+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:32.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:32.267+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:32.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:32.463+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:32.735 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:32 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:32.463+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:32.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:32.510+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:32.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:32.510+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:32.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:32.385+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:32.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:32 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:32.385+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:33.001 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:32 smithi098 podman[70458]: 2024-04-25 21:27:32.682673088 +0000 UTC m=+0.637754284 container remove 8014a000145e1016590d92cfda62ba3ec87bf327420c3711e83500e6c38bee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:33.285 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:33.145+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:33.285 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:33.145+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:33.285 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:32.906142074 +0000 UTC m=+0.018159538 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:33.285 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:33.033585032 +0000 UTC m=+0.145602529 container create 91817b2a4254e72eea5e412ed7008b40d783017f6d27af7618b0bbf42be4f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T21:27:33.385 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:33 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:33.092+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:33.579 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:33 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:33.504+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:33.579 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:33.285993284 +0000 UTC m=+0.398010739 container init 91817b2a4254e72eea5e412ed7008b40d783017f6d27af7618b0bbf42be4f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T21:27:33.579 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:33.289078494 +0000 UTC m=+0.401095950 container start 91817b2a4254e72eea5e412ed7008b40d783017f6d27af7618b0bbf42be4f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:27:33.580 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:33.403602296 +0000 UTC m=+0.515619764 container attach 91817b2a4254e72eea5e412ed7008b40d783017f6d27af7618b0bbf42be4f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T21:27:33.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:33.429+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:33.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:33.429+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:33.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:33.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:33.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:33.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:33.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:33 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:33.385+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:33.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:33 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:33.385+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:33.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:33 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:33.551+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:33.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:33 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:33.551+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:33.916 INFO:journalctl@ceph.osd.4.smithi098.stdout:Apr 25 21:27:33 smithi098 podman[70548]: 2024-04-25 21:27:33.581084295 +0000 UTC m=+0.693101742 container died 91817b2a4254e72eea5e412ed7008b40d783017f6d27af7618b0bbf42be4f7db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:34.256 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:34 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:34.194+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:34.256 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:34 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:34.194+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:34.277 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T21:27:34.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:34.316 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T21:27:34.316 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T21:27:34.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.5 2024-04-25T21:27:34.395 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 systemd[1]: Stopping Ceph osd.5 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:34.395 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:34.122+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:34.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:34 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:34.439+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:34.735 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:34 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:34.439+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:34.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:34 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:34.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:34.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:34 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:34.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:34.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:34.395+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:34.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:34.395+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:34.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:34.582+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:34.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:34.582+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:34.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:34.643+0000 7feca55da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:34.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:34.643+0000 7feca55da700 -1 osd.5 118 *** Got signal Terminated *** 2024-04-25T21:27:34.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:34 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:34.643+0000 7feca55da700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:34.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:34 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:34.548+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:35.418 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:35 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:35.145+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:35.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:35 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:35.225+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:35.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:35 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:35.225+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:35.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:35 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:35.451+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:35.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:35 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:35.451+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:35.536 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:35 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:35.181+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:35.536 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:35 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:35.181+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:35.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:35 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:35.419+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:35.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:35 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:35.419+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:35.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:35 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:35.611+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:35.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:35 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:35.611+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:35.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:35 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:35.536+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:36.426 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:36.127+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:36.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:36.204+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:36.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:36.204+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:36.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:36.441+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:36.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:36 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:36.441+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:36.504 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:36.177+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:36.505 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:36.177+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:36.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:36.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:36.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:36.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:36.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:36.571+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:36.840 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:36 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:36.571+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:36.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:36 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:36.505+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:37.398 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:37.129+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:37.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:37.201+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:37.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:37.201+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:37.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:37.409+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:37.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:37 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:37.409+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:37.512 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:37 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:37.170+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:37.512 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:37 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:37.170+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:37.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:37.552+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:37.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:37.552+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:37.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:37.399+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:37.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:37.399+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:37.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:37 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:37.399+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:37.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:37 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:37.513+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:38.423 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:38.152+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:38.423 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:38.152+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:38.423 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:38.152+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:38.502 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:38.116+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:38.503 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:38.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:38.503 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:38.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:38.503 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:38.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:38.546 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:38 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:38.175+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:38.546 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:38 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:38.175+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:38.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:38.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:38.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:38.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:38.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:38.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:38.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:38.503+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:38.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:38.503+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:38.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:38 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:38.503+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.186457+0000 front 2024-04-25T21:27:12.186482+0000 (oldest deadline 2024-04-25T21:27:38.086626+0000) 2024-04-25T21:27:38.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:38 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:38.547+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:38.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:38 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:38.547+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:39.473 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:39.111+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:39.474 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:39.324+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:39.474 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:39.324+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:39.474 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:39.324+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:39.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:39.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:39.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:39.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:39.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:39.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:39 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:39.424+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:39.558 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:39 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:39.202+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:39.559 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:39 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:39.202+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:39.559 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:39 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:39.202+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:39.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:39.474+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.983459+0000 front 2024-04-25T21:26:58.983582+0000 (oldest deadline 2024-04-25T21:27:22.483395+0000) 2024-04-25T21:27:39.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:39.474+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.584508+0000 front 2024-04-25T21:27:06.584706+0000 (oldest deadline 2024-04-25T21:27:28.284408+0000) 2024-04-25T21:27:39.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:39 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5[44746]: 2024-04-25T21:27:39.474+0000 7fec9dbc6700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.186457+0000 front 2024-04-25T21:27:12.186482+0000 (oldest deadline 2024-04-25T21:27:38.086626+0000) 2024-04-25T21:27:39.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:39 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:39.559+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:39.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:39 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:39.559+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:40.311 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:40 smithi177 podman[71987]: 2024-04-25 21:27:40.01780623 +0000 UTC m=+5.503550001 container died 1fad2a15e93513a17623576846fadded50dc4623bba98d27d468f16a96bc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:27:40.311 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:40 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:40.109+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:40.311 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:40 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:40.109+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:40.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:40.374+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:40.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:40.374+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:40.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:40.374+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:40.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:40.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:40.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:40.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:40.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:40 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:40.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:40.560 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:40 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:40.205+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:40.560 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:40 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:40.205+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:40.560 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:40 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:40.205+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:40.589 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:40 smithi177 podman[71987]: 2024-04-25 21:27:40.410996721 +0000 UTC m=+5.896740491 container cleanup 1fad2a15e93513a17623576846fadded50dc4623bba98d27d468f16a96bc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:40.589 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:40 smithi177 bash[71987]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5 2024-04-25T21:27:40.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:40 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:40.327+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:40.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:40 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:40.327+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:40.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:40 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:40.327+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:40.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:40 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:40.561+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:40.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:40 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:40.561+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:40.980 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:40 smithi177 podman[72004]: 2024-04-25 21:27:40.662304353 +0000 UTC m=+0.638624669 container remove 1fad2a15e93513a17623576846fadded50dc4623bba98d27d468f16a96bc0750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:27:41.265 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:41 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:41.138+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:41.266 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:41 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:41.138+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:41.266 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:40 smithi177 podman[72094]: 2024-04-25 21:27:40.883783464 +0000 UTC m=+0.017904383 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:41.266 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:41 smithi177 podman[72094]: 2024-04-25 21:27:41.013078903 +0000 UTC m=+0.147199806 container create 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:27:41.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:41.186+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:41.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:41.186+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:41.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:41.186+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:41.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:41.360+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:41.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:41.360+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:41.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:41.360+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:41.561 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:41 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:41.342+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:41.561 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:41 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:41.342+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:41.561 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:41 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:41.342+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:41.562 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:41 smithi177 podman[72094]: 2024-04-25 21:27:41.266969772 +0000 UTC m=+0.401090676 container init 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T21:27:41.562 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:41 smithi177 podman[72094]: 2024-04-25 21:27:41.270212132 +0000 UTC m=+0.404333038 container start 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:41.562 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:41 smithi177 podman[72094]: 2024-04-25 21:27:41.358536248 +0000 UTC m=+0.492657160 container attach 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T21:27:41.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:41 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:41.249+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:41.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:41 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:41.249+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:41.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:41 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:41.249+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:41.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:41 smithi177 podman[72094]: 2024-04-25 21:27:41.563026859 +0000 UTC m=+0.697147762 container died 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:27:41.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:41 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:41.595+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:41.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:41 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:41.595+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:42.155 INFO:journalctl@ceph.osd.5.smithi177.stdout:Apr 25 21:27:42 smithi177 podman[72120]: 2024-04-25 21:27:42.138750554 +0000 UTC m=+0.570196570 container remove 45130000b8e589bff163d37026d4adfa79f9b187a2f3305f85c7e6bc5a214159 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T21:27:42.162 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T21:27:42.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:42.199 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T21:27:42.199 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T21:27:42.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.6 2024-04-25T21:27:42.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:42.176+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:42.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:42.176+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:42.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:42.176+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:42.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 systemd[1]: Stopping Ceph osd.6 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:42.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.318+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:42.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.318+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:42.485 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.318+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:42.587 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:42 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:42.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:42.587 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:42 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:42.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:42.587 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:42 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:42.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:42.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:42 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:42.311+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:42.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:42 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:42.311+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:42.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:42 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:42.311+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:42.589 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:42 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:42.156+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:42.589 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:42 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:42.156+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:42.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:42 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:42.588+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:42.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:42 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:42.588+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:42.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.518+0000 7f5cdcdb1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:42.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.518+0000 7f5cdcdb1700 -1 osd.6 118 *** Got signal Terminated *** 2024-04-25T21:27:42.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:42 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:42.518+0000 7f5cdcdb1700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:43.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:43.301+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:43.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:43.301+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:43.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:43.301+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:43.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:43.136+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:43.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:43.136+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:43.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:43 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:43.136+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:43.589 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:43 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:43.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:43.589 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:43 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:43.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:43.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:43 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:43.336+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:43.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:43 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:43.336+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:43.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:43 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:43.336+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:43.625 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:43 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:43.238+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:43 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:43.238+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:43.626 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:43 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:43.238+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:43.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:43 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:43.626+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:43.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:43 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:43.626+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:44.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:44.075+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:44.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:44.075+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:44.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:44.272+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:44.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:44.272+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:44.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:44.272+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:44.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:44.174+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:44.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:44.174+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:44.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:44 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:44.174+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:44.649 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:44 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:44.272+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:44.650 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:44 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:44.272+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:44.650 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:44 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:44.272+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:44.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:44.383+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:44.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:44.383+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:44.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:44 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:44.383+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:44.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:44 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:44.650+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:44.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:44 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:44.650+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:45.376 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:45.107+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:45.376 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:45.107+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:45.376 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:45.107+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:45.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:45.286+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:45.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:45.286+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:45.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:45.286+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:45.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:45.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:45.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:45.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:45.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:45.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:45.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:45 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:45.183+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:45.607 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:45.242+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:45.608 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:45.242+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:45.608 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:45.242+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:45.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:45.376+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:45.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:45.376+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:45.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:45 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:45.376+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:45.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:45.608+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:45.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:45.608+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:45.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:45.608+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:46.357 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:46.098+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:46.357 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:46.098+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:46.357 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:46.098+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:46.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:46.241+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:46.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:46.241+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:46.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:46.241+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:46.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:46.147+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:46.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:46.147+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:46.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:46.147+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:46.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:46 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:46.147+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:46.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:46.224+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:46.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:46.224+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:46.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:46.224+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:46.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:46.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:46.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:46.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:46.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:46 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:46.358+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:46.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:46.619+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:46.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:46.619+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:46.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:46 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:46.619+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:47.400 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:47.116+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:47.401 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:47.116+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:47.401 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:47.116+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:47.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:47.244+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.718486+0000 front 2024-04-25T21:26:56.718514+0000 (oldest deadline 2024-04-25T21:27:20.818625+0000) 2024-04-25T21:27:47.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:47.244+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.920609+0000 front 2024-04-25T21:27:05.920452+0000 (oldest deadline 2024-04-25T21:27:31.820471+0000) 2024-04-25T21:27:47.484 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6[57771]: 2024-04-25T21:27:47.244+0000 7f5cd539d700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.821130+0000 front 2024-04-25T21:27:11.821214+0000 (oldest deadline 2024-04-25T21:27:37.721000+0000) 2024-04-25T21:27:47.484 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:47.194+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:47.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:47.194+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:47.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:47.194+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:47.485 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:47 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:47.194+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:47.580 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:47.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:47.580 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:47.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:47.580 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:47.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:47.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:47.401+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:47.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:47.401+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:47.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:47 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:47.401+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:47.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:47.580+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:47.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:47.580+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:47.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:47 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:47.580+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:48.293 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:47 smithi037 podman[109757]: 2024-04-25 21:27:47.997063885 +0000 UTC m=+5.595506646 container died 03253d294e2114ee5728fd7ea15ed1ae1b3cfef804fe7f60e792f1d3474e6f76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:27:48.294 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:48 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:48.226+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:48.294 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:48 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:48.226+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:48.294 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:48 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:48.226+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:48.294 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:48 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:48.226+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:48.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:48.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:48.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:48.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:48.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:48.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:48.551 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:48.245+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:48.551 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:48.245+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:48.551 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:48.245+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:48.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:48 smithi037 podman[109757]: 2024-04-25 21:27:48.494172068 +0000 UTC m=+6.092614834 container cleanup 03253d294e2114ee5728fd7ea15ed1ae1b3cfef804fe7f60e792f1d3474e6f76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:27:48.734 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:48 smithi037 bash[109757]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6 2024-04-25T21:27:48.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:48.429+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:48.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:48.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:48.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:48 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:48.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:48.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:48.552+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:48.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:48.552+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:48.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:48 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:48.552+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:49.074 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:48 smithi037 podman[109774]: 2024-04-25 21:27:48.754006692 +0000 UTC m=+0.750251162 container remove 03253d294e2114ee5728fd7ea15ed1ae1b3cfef804fe7f60e792f1d3474e6f76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:49.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:49.071+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:49.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:49.071+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:49.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:49.071+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:49.372 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:49 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:49.227+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:49 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:49.227+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:49 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:49.227+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:49 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:49.227+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:48.979260515 +0000 UTC m=+0.018437561 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:49.096610197 +0000 UTC m=+0.135787252 container create b52e6103fe3bc6bbddfd99e0476b377c347cacee9249ba4ddfa0feb9227003ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T21:27:49.373 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:49.374041445 +0000 UTC m=+0.413218459 container init b52e6103fe3bc6bbddfd99e0476b377c347cacee9249ba4ddfa0feb9227003ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T21:27:49.576 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:49.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:49.576 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:49.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:49.576 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:49.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:49.577 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:49.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:49.674 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:49.377825393 +0000 UTC m=+0.417002413 container start b52e6103fe3bc6bbddfd99e0476b377c347cacee9249ba4ddfa0feb9227003ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T21:27:49.675 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:49.49075823 +0000 UTC m=+0.529935246 container attach b52e6103fe3bc6bbddfd99e0476b377c347cacee9249ba4ddfa0feb9227003ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-25T21:27:49.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:49.392+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:49.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:49.392+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:49.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:49 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:49.392+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:49.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:49.577+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:49.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:49.577+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:49.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:49.577+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:49.984 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 25 21:27:49 smithi037 podman[109865]: 2024-04-25 21:27:49.676574838 +0000 UTC m=+0.715751863 container died b52e6103fe3bc6bbddfd99e0476b377c347cacee9249ba4ddfa0feb9227003ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:50.310 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:50 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:50.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:50.310 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:50 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:50.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:50.310 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:50 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:50.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:50.310 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:50 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:50.216+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:50.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T21:27:50.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:50.373 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T21:27:50.373 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T21:27:50.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.7 2024-04-25T21:27:50.396 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:50.112+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:50.396 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:50.112+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:50.396 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:50.112+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:50.612 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 systemd[1]: Stopping Ceph osd.7 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:50.612 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:50.222+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:50.612 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:50.222+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:50.612 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:50.222+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:50.613 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:50.222+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:50.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:50.397+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:50.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:50.397+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:50.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:50 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:50.397+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:50.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.613+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:50.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.613+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:50.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.613+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:50.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.694+0000 7f5234e5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:50.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.694+0000 7f5234e5a700 -1 osd.7 118 *** Got signal Terminated *** 2024-04-25T21:27:50.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:50 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:50.694+0000 7f5234e5a700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:51.390 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:51.125+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:51.390 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:51.125+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:51.390 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:51.125+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:51.565 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:51 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:51.250+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:51.565 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:51 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:51.250+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:51.565 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:51 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:51.250+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:51.566 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:51 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:51.250+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:51.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:51.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:51.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:51.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:51.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:51.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:51.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:51.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:51.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:51.391+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:51.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:51.391+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:51.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:51 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:51.391+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:51.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:51.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:51.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:51.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:51.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:51 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:51.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:52.434 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:52.095+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:52.435 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:52.095+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:52.435 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:52.095+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:52.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:52.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:52.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:52.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:52.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:52.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:52.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:52.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:52.640 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:52.252+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:52.705 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:52.435+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:52.706 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:52.435+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:52.706 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:52.435+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:52.706 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:52 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:52.435+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:52.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:52 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:52.284+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:52.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:52 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:52.284+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:52.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:52 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:52.285+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:52.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:52 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:52.285+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:52.911 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:52.640+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:52.911 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:52.640+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:52.911 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:52 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:52.640+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:53.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:53.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:53.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:53.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:53.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:53.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:53.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:53.080+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:53.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:53.235+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:53.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:53.235+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:53.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:53.235+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:53.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:53.235+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:53.641 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:53.235+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:53.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:53 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:53.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:53.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:53 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:53.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:53.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:53 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:53.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:53.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:53 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:53.256+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:53.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:53.409+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:53.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:53.409+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:53.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:53.409+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:53.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:53 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:53.409+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:53.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:53.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:53.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:53.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:53.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:53.641+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:54.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:54.059+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:54.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:54.059+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:54.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:54.059+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:54.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:54.059+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:54.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:54.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:54.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:54.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:54.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:54.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:54.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:54.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:54.594 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:54.227+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:54.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:54.236+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:54.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:54.236+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:54.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:54.236+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:54.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:54 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:54.236+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:54.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:54.386+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:54.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:54.386+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:54.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:54.386+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:54.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:54 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:54.386+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:54.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:54.594+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:54.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:54.594+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:54.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:54.594+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:54.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:54 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:54.594+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.669891+0000 front 2024-04-25T21:27:28.669849+0000 (oldest deadline 2024-04-25T21:27:54.570040+0000) 2024-04-25T21:27:55.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:55.061+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:55.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:55.061+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:55.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:55.061+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:55.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:55.061+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:55.619 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:55.220+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:55.619 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:55.220+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:55.620 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:55.220+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:55.620 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:55.220+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:55.620 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:55.220+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:55.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:55 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:55.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:55.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:55 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:55.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:55.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:55 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:55.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:55.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:55 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:55.251+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:55.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:55.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:55.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:55.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:55.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:55.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:55.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:55 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:55.427+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:55.916 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:55.620+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.764693+0000 front 2024-04-25T21:27:06.764666+0000 (oldest deadline 2024-04-25T21:27:28.965544+0000) 2024-04-25T21:27:55.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:55.620+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:15.367109+0000 front 2024-04-25T21:27:15.367187+0000 (oldest deadline 2024-04-25T21:27:37.667260+0000) 2024-04-25T21:27:55.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:55.620+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:21.668885+0000 front 2024-04-25T21:27:21.668968+0000 (oldest deadline 2024-04-25T21:27:45.168820+0000) 2024-04-25T21:27:55.917 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:55 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7[50862]: 2024-04-25T21:27:55.620+0000 7f523144e700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.669891+0000 front 2024-04-25T21:27:28.669849+0000 (oldest deadline 2024-04-25T21:27:54.570040+0000) 2024-04-25T21:27:56.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:56.042+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:56.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:56.042+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:56.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:56.042+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:56.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:56.042+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:56.416 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:56 smithi098 podman[70654]: 2024-04-25 21:27:56.15674557 +0000 UTC m=+5.595049551 container died ce430d8af64d296fee64670c2a21919c11565f4bfc159a967b30703d018eeb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:56.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:56 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:56.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:56.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:56 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:56.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:56.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:56 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:56.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:56.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:56 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:56.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:56.417 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:56 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:56.253+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:56.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:56 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:56.282+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:56.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:56 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:56.282+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:56.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:56 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:56.282+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:56.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:56 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:56.282+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:56.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:56 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:56.282+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:27:56.829 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:56 smithi098 podman[70654]: 2024-04-25 21:27:56.603054759 +0000 UTC m=+6.041358730 container cleanup ce430d8af64d296fee64670c2a21919c11565f4bfc159a967b30703d018eeb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T21:27:56.829 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:56 smithi098 bash[70654]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7 2024-04-25T21:27:56.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:56.473+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:56.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:56.473+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:56.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:56.473+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:56.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:56 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:56.473+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:57.166 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:56 smithi098 podman[70684]: 2024-04-25 21:27:56.829143078 +0000 UTC m=+0.662722157 container remove ce430d8af64d296fee64670c2a21919c11565f4bfc159a967b30703d018eeb08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:27:57.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:57.082+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:57.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:57.082+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:57.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:57.082+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:57.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:57.082+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.16964211 +0000 UTC m=+0.018180303 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.296826728 +0000 UTC m=+0.145364973 container create a5a933f54f2e009bc7a924862105b3e8962979ea96aa8fda4c39e6a5199e2fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:57.269+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:57.269+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:57.269+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:57.269+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:57.548 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:57.269+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:57.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:57 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:57.327+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:57.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:57 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:57.327+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:57.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:57 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:57.327+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:57.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:57 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:57.327+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:57.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:57 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:57.327+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:27:57.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:57.434+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:57.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:57.434+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:57.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:57.434+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:57.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:57 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:57.434+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:57.845 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.549284311 +0000 UTC m=+0.397822498 container init a5a933f54f2e009bc7a924862105b3e8962979ea96aa8fda4c39e6a5199e2fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T21:27:57.845 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.552589338 +0000 UTC m=+0.401127522 container start a5a933f54f2e009bc7a924862105b3e8962979ea96aa8fda4c39e6a5199e2fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T21:27:57.845 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.649314516 +0000 UTC m=+0.497852705 container attach a5a933f54f2e009bc7a924862105b3e8962979ea96aa8fda4c39e6a5199e2fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:27:58.166 INFO:journalctl@ceph.osd.7.smithi098.stdout:Apr 25 21:27:57 smithi098 podman[70774]: 2024-04-25 21:27:57.846232637 +0000 UTC m=+0.694770812 container died a5a933f54f2e009bc7a924862105b3e8962979ea96aa8fda4c39e6a5199e2fcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T21:27:58.466 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.117+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:58.466 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.117+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:58.466 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.117+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:58.466 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.117+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:58.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:58 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:58.306+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:58.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:58 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:58.306+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:58.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:58 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:58.306+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:58.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:58 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:58.306+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:58.519 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:58 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:58.306+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:58.540 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T21:27:58.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:27:58.580 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T21:27:58.580 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T21:27:58.580 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.8 2024-04-25T21:27:58.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:58.279+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:58.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:58.279+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:58.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:58.279+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:58.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:58.279+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:58.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:58 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:58.279+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:27:58.839 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 systemd[1]: Stopping Ceph osd.8 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:27:58.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:58.466+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:58.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:58.466+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:58.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:58.466+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:58.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:58.466+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:27:59.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.899+0000 7fe6fe55c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:27:59.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.899+0000 7fe6fe55c700 -1 osd.8 118 *** Got signal Terminated *** 2024-04-25T21:27:59.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:58 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:58.899+0000 7fe6fe55c700 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:27:59.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:59.090+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:27:59.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:59.090+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:27:59.340 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:59.090+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:27:59.340 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:27:59.090+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:27:59.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:59 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:59.328+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:27:59.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:59 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:59.328+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:27:59.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:59 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:59.328+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:27:59.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:59 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:59.328+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:27:59.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:27:59 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:27:59.328+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:27:59.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:59.286+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:27:59.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:59.286+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:27:59.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:59.286+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:27:59.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:59.286+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:27:59.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:27:59 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:27:59.286+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:27:59.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:59.437+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:27:59.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:59.437+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:27:59.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:59.437+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:27:59.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:27:59 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:27:59.437+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:00.444 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:00.115+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:28:00.445 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:00.115+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:28:00.445 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:00.115+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:28:00.445 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:00.115+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:28:00.635 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:00.323+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:00.636 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:00.323+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:00.636 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:00.323+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:00.636 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:00.323+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:00.636 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:00 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:00.323+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:00.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:00 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:00.333+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:00.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:00 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:00.333+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:00.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:00 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:00.333+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:00.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:00 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:00.333+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:00.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:00 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:00.333+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:00.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:00.445+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:00.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:00.445+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:00.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:00.445+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:00.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:00 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:00.445+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:01.400 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:01.134+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:28:01.400 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:01.134+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:28:01.400 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:01.134+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:28:01.400 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:01.134+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:01.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:01 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:01.308+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:01.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:01.295+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:01.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:01.295+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:01.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:01.295+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:01.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:01.295+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:01.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:01 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:01.295+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:01.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:01.400+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:01.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:01.400+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:01.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:01.400+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:01.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:01.400+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:02.430 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:02.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:28:02.430 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:02.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:28:02.430 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:02.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:28:02.430 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:02.114+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:02.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:02 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:02.310+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:02.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:02 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:02.249+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:02.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:02.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:02.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:02.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:02.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:02.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:02.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:02.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:02.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:02 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:02.430+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:03.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:03.077+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:59.207294+0000 front 2024-04-25T21:26:59.207192+0000 (oldest deadline 2024-04-25T21:27:21.407675+0000) 2024-04-25T21:28:03.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:03.077+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.010311+0000 front 2024-04-25T21:27:13.010351+0000 (oldest deadline 2024-04-25T21:27:39.410941+0000) 2024-04-25T21:28:03.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:03.077+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:19.411555+0000 front 2024-04-25T21:27:19.411476+0000 (oldest deadline 2024-04-25T21:27:44.711350+0000) 2024-04-25T21:28:03.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:03.077+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.812441+0000 front 2024-04-25T21:27:28.812390+0000 (oldest deadline 2024-04-25T21:27:52.213069+0000) 2024-04-25T21:28:03.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8[51747]: 2024-04-25T21:28:03.077+0000 7fe6f6b48700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.214306+0000 front 2024-04-25T21:27:39.214256+0000 (oldest deadline 2024-04-25T21:28:02.714435+0000) 2024-04-25T21:28:03.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:03.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:03.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:03.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:03.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:03.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:03 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:03.289+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:03.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:03.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:03.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:03.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:03.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:03.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:03 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:03.289+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:03.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:03.479+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:03.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:03.479+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:03.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:03.479+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:03.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:03.479+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:03.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:03 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:03.479+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:04.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:04 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:04.325+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:04.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:04.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:04.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:04.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:04.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:04.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:04 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:04.324+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:04.749 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:04 smithi177 podman[72210]: 2024-04-25 21:28:04.428272593 +0000 UTC m=+5.649666820 container died 022c6212544b98f29ad9ca4d34eed3a073adbab6709194ad7f4032179d3b4b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T21:28:04.749 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:04 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:04.469+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:04.749 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:04 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:04.469+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:04.749 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:04 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:04.469+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:04.749 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:04 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:04.469+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:04.750 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:04 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:04.469+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:05.058 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:04 smithi177 podman[72210]: 2024-04-25 21:28:04.849374739 +0000 UTC m=+6.070768953 container cleanup 022c6212544b98f29ad9ca4d34eed3a073adbab6709194ad7f4032179d3b4b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T21:28:05.058 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:04 smithi177 bash[72210]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8 2024-04-25T21:28:05.339 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72227]: 2024-04-25 21:28:05.059135663 +0000 UTC m=+0.629104504 container remove 022c6212544b98f29ad9ca4d34eed3a073adbab6709194ad7f4032179d3b4b7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:05.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:05 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:05.353+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:05.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:05.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:05.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:05.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:05.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:05.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:05 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:05.332+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:05.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:05.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:05.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:05.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:05.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:05.744 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72320]: 2024-04-25 21:28:05.371825304 +0000 UTC m=+0.018097751 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:28:05.745 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72320]: 2024-04-25 21:28:05.493160023 +0000 UTC m=+0.139432449 container create 27d146be6620ff8af1033c9399f8e50055eab763fef68dfc9e9c87c151098ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T21:28:05.745 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72320]: 2024-04-25 21:28:05.745390938 +0000 UTC m=+0.391663360 container init 27d146be6620ff8af1033c9399f8e50055eab763fef68dfc9e9c87c151098ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:28:06.036 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72320]: 2024-04-25 21:28:05.748573347 +0000 UTC m=+0.394845770 container start 27d146be6620ff8af1033c9399f8e50055eab763fef68dfc9e9c87c151098ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:28:06.036 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:05 smithi177 podman[72320]: 2024-04-25 21:28:05.845815465 +0000 UTC m=+0.492087913 container attach 27d146be6620ff8af1033c9399f8e50055eab763fef68dfc9e9c87c151098ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T21:28:06.312 INFO:journalctl@ceph.osd.8.smithi177.stdout:Apr 25 21:28:06 smithi177 podman[72320]: 2024-04-25 21:28:06.037906401 +0000 UTC m=+0.684178839 container died 27d146be6620ff8af1033c9399f8e50055eab763fef68dfc9e9c87c151098ada (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T21:28:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:06.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:06.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:06.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:06.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:06 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:06.507+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:06.657 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T21:28:06.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:06.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:06.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:06.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:06.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:06.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:06 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:06.327+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:06.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:06.695 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T21:28:06.695 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T21:28:06.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.9 2024-04-25T21:28:06.723 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:06.724 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:06.724 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:06.724 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:06.724 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:06.724 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:06.328+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:06.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:06 smithi037 systemd[1]: Stopping Ceph osd.9 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.004+0000 7f1e1713c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.004+0000 7f1e1713c700 -1 osd.9 118 *** Got signal Terminated *** 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.004+0000 7f1e1713c700 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:07.340 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:07.341 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:07.341 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:07.341 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:07 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:07.340+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:07.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:07 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:07.477+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:07.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:07 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:07.477+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:07.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:07 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:07.477+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:07.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:07 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:07.477+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:07.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:07 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:07.477+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:07.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:07 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:07.348+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:08.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:08.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:08.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:08.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:08.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:08.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:08 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:08.317+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:08.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:08 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:08.369+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:08.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:08.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:08.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:08.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:08.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:08.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:08.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:08.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:08.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:08 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:08.506+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:09.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:09 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:09.295+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:09.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:09.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:09.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:09.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:09.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:09.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:09 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:09.333+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:09.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:09.529+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:09.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:09.529+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:09.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:09.529+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:09.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:09.529+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:09.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:09.529+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:10.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:10 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:10.315+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:10.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:10.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:10.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:10.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:10.690 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:10.691 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:10 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:10.320+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:10.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:10.572+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:10.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:10.572+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:10.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:10.572+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:10.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:10.572+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:10.840 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:10 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:10.572+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:11.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:11.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:11 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:11.359+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:11.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:58.058366+0000 front 2024-04-25T21:26:58.058527+0000 (oldest deadline 2024-04-25T21:27:22.758447+0000) 2024-04-25T21:28:11.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.259547+0000 front 2024-04-25T21:27:06.259384+0000 (oldest deadline 2024-04-25T21:27:31.559457+0000) 2024-04-25T21:28:11.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:11.560186+0000 front 2024-04-25T21:27:11.560225+0000 (oldest deadline 2024-04-25T21:27:37.459973+0000) 2024-04-25T21:28:11.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.661220+0000 front 2024-04-25T21:27:22.661284+0000 (oldest deadline 2024-04-25T21:27:44.361446+0000) 2024-04-25T21:28:11.734 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:30.663700+0000 front 2024-04-25T21:27:30.663681+0000 (oldest deadline 2024-04-25T21:27:55.963457+0000) 2024-04-25T21:28:11.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:35.964283+0000 front 2024-04-25T21:27:35.964214+0000 (oldest deadline 2024-04-25T21:28:01.864132+0000) 2024-04-25T21:28:11.735 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:11 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9[64671]: 2024-04-25T21:28:11.341+0000 7f1e13730700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:47.165541+0000 front 2024-04-25T21:27:47.165587+0000 (oldest deadline 2024-04-25T21:28:10.665707+0000) 2024-04-25T21:28:11.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:11 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:11.547+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:11.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:11 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:11.547+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:11.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:11 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:11.547+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:11.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:11 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:11.547+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:11.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:11 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:11.547+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:12.663 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:12 smithi037 podman[109981]: 2024-04-25 21:28:12.389228066 +0000 UTC m=+5.492004213 container died 55cc52ec43b7d10b00f3c4e274aba65270af7457218ccad08083934c9a9d07a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:12.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:12.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:12.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:12.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:12.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:12.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:12.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:12 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:12.351+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:12.839 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:12 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:12.554+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:12.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:12 smithi037 podman[109981]: 2024-04-25 21:28:12.738458454 +0000 UTC m=+5.841234602 container cleanup 55cc52ec43b7d10b00f3c4e274aba65270af7457218ccad08083934c9a9d07a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:12.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:12 smithi037 bash[109981]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9 2024-04-25T21:28:13.424 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[109999]: 2024-04-25 21:28:13.072960637 +0000 UTC m=+0.681628148 container remove 55cc52ec43b7d10b00f3c4e274aba65270af7457218ccad08083934c9a9d07a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:13.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:13 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:13.319+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:13.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.328354238 +0000 UTC m=+0.018299052 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:28:13.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.432263137 +0000 UTC m=+0.122207945 container create 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:13.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.659706385 +0000 UTC m=+0.349651188 container init 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:28:13.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.662928425 +0000 UTC m=+0.352873228 container start 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:28:13.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.726165391 +0000 UTC m=+0.416110200 container attach 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T21:28:13.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:13 smithi037 podman[110088]: 2024-04-25 21:28:13.965069308 +0000 UTC m=+0.655014109 container died 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:14.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:13.603+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:14.517 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 25 21:28:14 smithi037 podman[110114]: 2024-04-25 21:28:14.504399794 +0000 UTC m=+0.533513089 container remove 187253dd74ee07d56e9b0112511c0dc5161053f46f3f381f35208bfd8bf62e3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T21:28:14.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T21:28:14.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:14.603 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T21:28:14.603 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T21:28:14.603 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.10 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:14.628 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:14.897 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 systemd[1]: Stopping Ceph osd.10 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:15.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:14 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:14.613+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:15.166 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.897+0000 7f34c2f91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:15.167 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.897+0000 7f34c2f91700 -1 osd.10 118 *** Got signal Terminated *** 2024-04-25T21:28:15.167 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:14 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:14.897+0000 7f34c2f91700 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:28:15.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:15.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:15 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:15.394+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:16.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:16.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:16.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:16.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:16.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:16.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:15 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:15.600+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:16.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:16.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:16 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:16.364+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:17.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:17.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:17.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:17.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:17.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:17.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:16 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:16.626+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:17.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:17.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:17 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:17.374+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:18.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:18.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:18.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:18.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:18.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:18.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:17 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:17.647+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:18.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:18 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:18.363+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:19.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:19.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:19.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:19.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:19.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:19.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:18 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:18.643+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:19.666 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:56.031762+0000 front 2024-04-25T21:26:56.031922+0000 (oldest deadline 2024-04-25T21:27:21.331694+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:06.632668+0000 front 2024-04-25T21:27:06.632808+0000 (oldest deadline 2024-04-25T21:27:28.332968+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:12.433873+0000 front 2024-04-25T21:27:12.433984+0000 (oldest deadline 2024-04-25T21:27:38.333684+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-25T21:27:22.836037+0000 front 2024-04-25T21:27:22.835907+0000 (oldest deadline 2024-04-25T21:27:48.735837+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:28.736297+0000 front 2024-04-25T21:27:28.736229+0000 (oldest deadline 2024-04-25T21:27:52.236400+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.238080+0000 front 2024-04-25T21:27:39.238048+0000 (oldest deadline 2024-04-25T21:28:00.337997+0000) 2024-04-25T21:28:19.667 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:19 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10[57749]: 2024-04-25T21:28:19.343+0000 7f34bf585700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:42.539720+0000 front 2024-04-25T21:27:42.539751+0000 (oldest deadline 2024-04-25T21:28:08.439540+0000) 2024-04-25T21:28:20.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:20.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:20.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:20.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:20.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:20.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:20.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:19 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:19.652+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:20.606 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:20 smithi098 podman[70883]: 2024-04-25 21:28:20.339105429 +0000 UTC m=+5.541314082 container died 3b717ebaf07b2e9b91c3408b443c078fb1f06f9d554aa7a6db0c95395bc9c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-25T21:28:20.916 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:20 smithi098 podman[70883]: 2024-04-25 21:28:20.706698174 +0000 UTC m=+5.908906888 container cleanup 3b717ebaf07b2e9b91c3408b443c078fb1f06f9d554aa7a6db0c95395bc9c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:28:20.916 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:20 smithi098 bash[70883]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:21.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:20 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:20.685+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:21.300 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:20 smithi098 podman[70901]: 2024-04-25 21:28:20.974321973 +0000 UTC m=+0.633309352 container remove 3b717ebaf07b2e9b91c3408b443c078fb1f06f9d554aa7a6db0c95395bc9c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T21:28:21.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.204709281 +0000 UTC m=+0.017786209 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:28:21.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.308458044 +0000 UTC m=+0.121535005 container create e3692c36e7662e4ee2f03bbc17ca99ab9c52f58baabf2d81da614e8846097130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:21.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.535787537 +0000 UTC m=+0.348864451 container init e3692c36e7662e4ee2f03bbc17ca99ab9c52f58baabf2d81da614e8846097130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T21:28:21.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.538951802 +0000 UTC m=+0.352028726 container start e3692c36e7662e4ee2f03bbc17ca99ab9c52f58baabf2d81da614e8846097130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T21:28:21.618 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.619068931 +0000 UTC m=+0.432145862 container attach e3692c36e7662e4ee2f03bbc17ca99ab9c52f58baabf2d81da614e8846097130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:28:21.916 INFO:journalctl@ceph.osd.10.smithi098.stdout:Apr 25 21:28:21 smithi098 podman[70994]: 2024-04-25 21:28:21.832173093 +0000 UTC m=+0.645250000 container died e3692c36e7662e4ee2f03bbc17ca99ab9c52f58baabf2d81da614e8846097130 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:28:22.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.734+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:22.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.734+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:22.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.734+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:22.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.734+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:22.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.734+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:22.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:22.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:21 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:21.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:22.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T21:28:22.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:22.485 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T21:28:22.485 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T21:28:22.486 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.11 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 systemd[1]: Stopping Ceph osd.11 for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.749+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:22.784 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.784+0000 7fc35feaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:22.785 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.784+0000 7fc35feaf700 -1 osd.11 118 *** Got signal Terminated *** 2024-04-25T21:28:22.785 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:22 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:22.784+0000 7fc35feaf700 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:28:24.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:24.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:24.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:24.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:24.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:24.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:24.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:23 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:23.772+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:25.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:25.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:25.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:25.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:25.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:25.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:25.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:24 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:24.735+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:26.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:26.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:26.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:26.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:26.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:26.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:26.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:25 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:25.759+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:27.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:27.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:27.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:27.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:27.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:27.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:27.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:27.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:26 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:26.711+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-04-25T21:28:02.204744+0000 front 2024-04-25T21:28:02.204771+0000 (oldest deadline 2024-04-25T21:28:26.304745+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-25T21:26:57.492672+0000 front 2024-04-25T21:26:57.492706+0000 (oldest deadline 2024-04-25T21:27:25.693141+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6804 osd.1 since back 2024-04-25T21:27:05.694147+0000 front 2024-04-25T21:27:05.694271+0000 (oldest deadline 2024-04-25T21:27:29.193602+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2024-04-25T21:27:13.795056+0000 front 2024-04-25T21:27:13.795094+0000 (oldest deadline 2024-04-25T21:27:36.695153+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6812 osd.4 since back 2024-04-25T21:27:27.697498+0000 front 2024-04-25T21:27:27.697606+0000 (oldest deadline 2024-04-25T21:27:52.397679+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2024-04-25T21:27:39.399141+0000 front 2024-04-25T21:27:39.399194+0000 (oldest deadline 2024-04-25T21:28:01.699192+0000) 2024-04-25T21:28:28.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-25T21:27:46.101640+0000 front 2024-04-25T21:27:46.101589+0000 (oldest deadline 2024-04-25T21:28:11.902177+0000) 2024-04-25T21:28:28.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.98:6820 osd.7 since back 2024-04-25T21:27:54.703701+0000 front 2024-04-25T21:27:54.703620+0000 (oldest deadline 2024-04-25T21:28:19.403510+0000) 2024-04-25T21:28:28.090 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:27 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11[58526]: 2024-04-25T21:28:27.743+0000 7fc35849b700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.177:6820 osd.8 since back 2024-04-25T21:28:02.204744+0000 front 2024-04-25T21:28:02.204771+0000 (oldest deadline 2024-04-25T21:28:26.304745+0000) 2024-04-25T21:28:28.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:28 smithi177 podman[72438]: 2024-04-25 21:28:28.323429527 +0000 UTC m=+5.645029624 container died 62a6ac73c676a83aa6411376263c786562d6b97bc6eb630049c3c4bf223cc807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T21:28:28.960 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:28 smithi177 podman[72438]: 2024-04-25 21:28:28.677674193 +0000 UTC m=+5.999274290 container cleanup 62a6ac73c676a83aa6411376263c786562d6b97bc6eb630049c3c4bf223cc807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T21:28:28.961 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:28 smithi177 bash[72438]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11 2024-04-25T21:28:29.301 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:28 smithi177 podman[72468]: 2024-04-25 21:28:28.962088109 +0000 UTC m=+0.632343053 container remove 62a6ac73c676a83aa6411376263c786562d6b97bc6eb630049c3c4bf223cc807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T21:28:29.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.205540322 +0000 UTC m=+0.017930072 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:28:29.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.304675556 +0000 UTC m=+0.117065291 container create a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:28:29.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.523669573 +0000 UTC m=+0.336059308 container init a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T21:28:29.590 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.527237574 +0000 UTC m=+0.339627318 container start a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:28:30.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.618128756 +0000 UTC m=+0.430518502 container attach a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T21:28:30.089 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:29 smithi177 podman[72563]: 2024-04-25 21:28:29.815625737 +0000 UTC m=+0.628015474 container died a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T21:28:30.411 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T21:28:30.434 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:30 smithi177 podman[72590]: 2024-04-25 21:28:30.39329169 +0000 UTC m=+0.572060702 container remove a0dda5fac937e7deb3297218c08378c858c2718679704202e0e4ddf34f058f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T21:28:30.434 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:30 smithi177 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-25T21:28:30.434 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:30 smithi177 systemd[1]: Stopped Ceph osd.11 for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:28:30.434 INFO:journalctl@ceph.osd.11.smithi177.stdout:Apr 25 21:28:30 smithi177 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@osd.11.service: Consumed 11.737s CPU time. 2024-04-25T21:28:30.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:30.450 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T21:28:30.450 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T21:28:30.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.a 2024-04-25T21:28:30.734 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:30 smithi037 systemd[1]: Stopping Ceph mds.a for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:31.010 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: 2024-04-25T21:28:30.763+0000 7fda63e7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:31.011 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:30 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a[72179]: 2024-04-25T21:28:30.763+0000 7fda63e7e700 -1 mds.a *** got signal Terminated *** 2024-04-25T21:28:32.896 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:32 smithi037 podman[110193]: 2024-04-25 21:28:32.565918409 +0000 UTC m=+1.915641089 container died d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:28:32.896 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:32 smithi037 podman[110193]: 2024-04-25 21:28:32.897733546 +0000 UTC m=+2.247456223 container cleanup d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T21:28:33.234 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:32 smithi037 bash[110193]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a 2024-04-25T21:28:33.234 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 25 21:28:33 smithi037 podman[110219]: 2024-04-25 21:28:33.090550757 +0000 UTC m=+0.526999222 container remove d73561b617b32fcc23de2ce1c5c0f64efcead87463911107b7e12cb08f7000e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:28:33.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T21:28:33.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:33.368 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T21:28:33.368 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T21:28:33.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.d 2024-04-25T21:28:33.684 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:33 smithi037 systemd[1]: Stopping Ceph mds.d for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:33.984 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d[70889]: 2024-04-25T21:28:33.684+0000 7fed24d46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:33.984 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:33 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d[70889]: 2024-04-25T21:28:33.684+0000 7fed24d46700 -1 mds.d *** got signal Terminated *** 2024-04-25T21:28:37.667 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:37 smithi037 podman[110327]: 2024-04-25 21:28:37.403409285 +0000 UTC m=+3.815500349 container died bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T21:28:37.926 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:37 smithi037 podman[110327]: 2024-04-25 21:28:37.743124899 +0000 UTC m=+4.155215960 container cleanup bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:28:37.926 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:37 smithi037 bash[110327]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d 2024-04-25T21:28:38.177 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T21:28:38.199 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:37 smithi037 podman[110342]: 2024-04-25 21:28:37.927455343 +0000 UTC m=+0.513285545 container remove bbdc59c9a89a62e1ae89427f6077f94a666291355bd03bce5de2fe885e64cb72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T21:28:38.199 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:38 smithi037 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.d.service: Deactivated successfully. 2024-04-25T21:28:38.199 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:38 smithi037 systemd[1]: Stopped Ceph mds.d for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:28:38.199 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 25 21:28:38 smithi037 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.259s CPU time. 2024-04-25T21:28:38.211 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:38.211 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T21:28:38.211 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T21:28:38.211 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.g 2024-04-25T21:28:38.484 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:38 smithi037 systemd[1]: Stopping Ceph mds.g for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:38.984 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: 2024-04-25T21:28:38.588+0000 7f9aa20dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:38.984 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:38 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g[71808]: 2024-04-25T21:28:38.588+0000 7f9aa20dc700 -1 mds.g *** got signal Terminated *** 2024-04-25T21:28:40.855 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:40 smithi037 podman[110450]: 2024-04-25 21:28:40.558577616 +0000 UTC m=+2.126247747 container died 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:28:41.196 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:40 smithi037 podman[110450]: 2024-04-25 21:28:40.938827042 +0000 UTC m=+2.506497172 container cleanup 32bd1f352d9a8877bdbd6afb7c38b5ce390f5cd946777b1764d68d938907348d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T21:28:41.197 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 25 21:28:40 smithi037 bash[110450]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-g 2024-04-25T21:28:41.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T21:28:41.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:41.404 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T21:28:41.404 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T21:28:41.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.j 2024-04-25T21:28:41.734 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:41 smithi037 systemd[1]: Stopping Ceph mds.j for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:42.049 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j[70424]: 2024-04-25T21:28:41.758+0000 7f035717e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:42.050 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:41 smithi037 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j[70424]: 2024-04-25T21:28:41.759+0000 7f035717e700 -1 mds.j *** got signal Terminated *** 2024-04-25T21:28:44.917 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:44 smithi037 podman[110572]: 2024-04-25 21:28:44.583252072 +0000 UTC m=+2.947878718 container died 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T21:28:45.201 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 podman[110572]: 2024-04-25 21:28:45.017966644 +0000 UTC m=+3.382593290 container cleanup 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:45.201 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 bash[110572]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j 2024-04-25T21:28:45.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T21:28:45.464 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 podman[110586]: 2024-04-25 21:28:45.202187484 +0000 UTC m=+0.609912262 container remove 7511b9ccc4336e9c1c03c1ee0fc01e5cf1c1730ee071a4607cff0dbc04f7b0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:28:45.464 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-25T21:28:45.464 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 systemd[1]: Stopped Ceph mds.j for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:28:45.464 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 25 21:28:45 smithi037 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.212s CPU time. 2024-04-25T21:28:45.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:45.475 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T21:28:45.475 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T21:28:45.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.b 2024-04-25T21:28:45.807 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:45 smithi098 systemd[1]: Stopping Ceph mds.b for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:46.166 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b[62963]: 2024-04-25T21:28:45.806+0000 7fa50f46a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:46.166 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:45 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b[62963]: 2024-04-25T21:28:45.806+0000 7fa50f46a700 -1 mds.b *** got signal Terminated *** 2024-04-25T21:28:48.907 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:48 smithi098 podman[71112]: 2024-04-25 21:28:48.586983296 +0000 UTC m=+2.911369973 container died 5be65f57b5a3ac4a1e89db6ec268a0bf43c29ff878988e2d094e6658a92e1309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T21:28:49.166 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:48 smithi098 podman[71112]: 2024-04-25 21:28:48.990402335 +0000 UTC m=+3.314789008 container cleanup 5be65f57b5a3ac4a1e89db6ec268a0bf43c29ff878988e2d094e6658a92e1309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:28:49.167 INFO:journalctl@ceph.mds.b.smithi098.stdout:Apr 25 21:28:48 smithi098 bash[71112]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-b 2024-04-25T21:28:49.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T21:28:49.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:49.449 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T21:28:49.449 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T21:28:49.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.e 2024-04-25T21:28:49.768 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:49 smithi098 systemd[1]: Stopping Ceph mds.e for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:50.166 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: 2024-04-25T21:28:49.768+0000 7f425856d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:50.167 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:49 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e[62340]: 2024-04-25T21:28:49.768+0000 7f425856d700 -1 mds.e *** got signal Terminated *** 2024-04-25T21:28:52.843 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:52 smithi098 podman[71234]: 2024-04-25 21:28:52.57659799 +0000 UTC m=+2.912047489 container died 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T21:28:53.166 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:52 smithi098 podman[71234]: 2024-04-25 21:28:52.94408751 +0000 UTC m=+3.279537020 container cleanup 881fea7f9c98fab7d24250d36a0a569763ccf53f75137d4a79cd2f0bee2633bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:28:53.166 INFO:journalctl@ceph.mds.e.smithi098.stdout:Apr 25 21:28:52 smithi098 bash[71234]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-e 2024-04-25T21:28:53.370 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T21:28:53.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:53.402 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T21:28:53.402 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T21:28:53.402 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.h 2024-04-25T21:28:53.722 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:53 smithi098 systemd[1]: Stopping Ceph mds.h for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:54.166 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: 2024-04-25T21:28:53.722+0000 7f958391e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:54.167 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:53 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h[62652]: 2024-04-25T21:28:53.722+0000 7f958391e700 -1 mds.h *** got signal Terminated *** 2024-04-25T21:28:56.882 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:56 smithi098 podman[71354]: 2024-04-25 21:28:56.570841447 +0000 UTC m=+2.947732567 container died 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T21:28:57.166 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:56 smithi098 podman[71354]: 2024-04-25 21:28:56.96505085 +0000 UTC m=+3.341941969 container cleanup 1e1d77ab28c62f1b11e47167c6b34bf292fb24066832154b2fff032dfb549469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:28:57.166 INFO:journalctl@ceph.mds.h.smithi098.stdout:Apr 25 21:28:56 smithi098 bash[71354]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-h 2024-04-25T21:28:57.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T21:28:57.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:28:57.373 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T21:28:57.373 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T21:28:57.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.k 2024-04-25T21:28:57.666 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:28:57 smithi098 systemd[1]: Stopping Ceph mds.k for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:28:58.166 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:28:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: 2024-04-25T21:28:57.711+0000 7f57bcc4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:28:58.167 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:28:57 smithi098 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k[62023]: 2024-04-25T21:28:57.711+0000 7f57bcc4e700 -1 mds.k *** got signal Terminated *** 2024-04-25T21:29:00.916 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:29:00 smithi098 podman[71486]: 2024-04-25 21:29:00.579951951 +0000 UTC m=+2.981392874 container died e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:29:01.227 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:29:01 smithi098 podman[71486]: 2024-04-25 21:29:01.027352273 +0000 UTC m=+3.428793202 container cleanup e3ed99f7701bcc951ce9eecf6985eb034a287cab03dcdcf0df9256fff4c21702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T21:29:01.227 INFO:journalctl@ceph.mds.k.smithi098.stdout:Apr 25 21:29:01 smithi098 bash[71486]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-k 2024-04-25T21:29:01.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T21:29:01.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:29:01.484 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T21:29:01.484 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T21:29:01.485 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.c 2024-04-25T21:29:01.783 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:01 smithi177 systemd[1]: Stopping Ceph mds.c for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:29:02.038 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: 2024-04-25T21:29:01.783+0000 7ff9b7d82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:29:02.038 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:01 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c[63435]: 2024-04-25T21:29:01.783+0000 7ff9b7d82700 -1 mds.c *** got signal Terminated *** 2024-04-25T21:29:04.839 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:04 smithi177 podman[72668]: 2024-04-25 21:29:04.578712486 +0000 UTC m=+2.906993086 container died 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T21:29:05.260 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:04 smithi177 podman[72668]: 2024-04-25 21:29:04.841206069 +0000 UTC m=+3.169486669 container cleanup 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T21:29:05.260 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:04 smithi177 bash[72668]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c 2024-04-25T21:29:05.260 INFO:journalctl@ceph.mds.c.smithi177.stdout:Apr 25 21:29:05 smithi177 podman[72694]: 2024-04-25 21:29:05.050818893 +0000 UTC m=+0.472381863 container remove 3770701c39239105e32c4e34ba8768394aaaf9209fe9b6f7cad4a308c8bf62d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T21:29:05.284 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T21:29:05.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:29:05.319 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T21:29:05.319 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T21:29:05.319 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.f 2024-04-25T21:29:05.590 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:05 smithi177 systemd[1]: Stopping Ceph mds.f for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:29:06.089 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: 2024-04-25T21:29:05.660+0000 7f4916876700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:29:06.089 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:05 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f[63126]: 2024-04-25T21:29:05.660+0000 7f4916876700 -1 mds.f *** got signal Terminated *** 2024-04-25T21:29:08.839 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:08 smithi177 podman[72800]: 2024-04-25 21:29:08.575754039 +0000 UTC m=+3.036096686 container died c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T21:29:09.194 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:08 smithi177 podman[72800]: 2024-04-25 21:29:08.936247404 +0000 UTC m=+3.396590058 container cleanup c1e687316958b85cf76e782ebd9c3c88d5a32c46a8e2383a18ddf45445e988a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:29:09.195 INFO:journalctl@ceph.mds.f.smithi177.stdout:Apr 25 21:29:08 smithi177 bash[72800]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-f 2024-04-25T21:29:09.362 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T21:29:09.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:29:09.395 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T21:29:09.395 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T21:29:09.395 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.i 2024-04-25T21:29:09.714 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:09 smithi177 systemd[1]: Stopping Ceph mds.i for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:29:10.089 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: 2024-04-25T21:29:09.713+0000 7f72b2839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:29:10.089 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:09 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i[63738]: 2024-04-25T21:29:09.713+0000 7f72b2839700 -1 mds.i *** got signal Terminated *** 2024-04-25T21:29:12.839 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:12 smithi177 podman[72922]: 2024-04-25 21:29:12.583280102 +0000 UTC m=+2.973752340 container died 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T21:29:13.148 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:12 smithi177 podman[72922]: 2024-04-25 21:29:12.931707771 +0000 UTC m=+3.322180042 container cleanup 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:29:13.148 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:12 smithi177 bash[72922]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i 2024-04-25T21:29:13.394 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T21:29:13.413 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:13 smithi177 podman[72936]: 2024-04-25 21:29:13.149457844 +0000 UTC m=+0.560296229 container remove 3e478684a0363d1113f93ece442358a8d1dd508198f9e5778a6665fddd5be320 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T21:29:13.414 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:13 smithi177 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T21:29:13.414 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:13 smithi177 systemd[1]: Stopped Ceph mds.i for 1788dafa-0346-11ef-bc93-c7b262605968. 2024-04-25T21:29:13.414 INFO:journalctl@ceph.mds.i.smithi177.stdout:Apr 25 21:29:13 smithi177 systemd[1]: ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.i.service: Consumed 14.731s CPU time. 2024-04-25T21:29:13.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:29:13.425 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T21:29:13.425 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T21:29:13.425 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.l 2024-04-25T21:29:13.767 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:13 smithi177 systemd[1]: Stopping Ceph mds.l for 1788dafa-0346-11ef-bc93-c7b262605968... 2024-04-25T21:29:14.089 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: 2024-04-25T21:29:13.768+0000 7ff0a18b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:29:14.089 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:13 smithi177 ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l[62817]: 2024-04-25T21:29:13.768+0000 7ff0a18b6700 -1 mds.l *** got signal Terminated *** 2024-04-25T21:29:16.839 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:16 smithi177 podman[73044]: 2024-04-25 21:29:16.58261184 +0000 UTC m=+2.944421707 container died 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:29:17.194 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:16 smithi177 podman[73044]: 2024-04-25 21:29:16.859795007 +0000 UTC m=+3.221604890 container cleanup 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T21:29:17.194 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:16 smithi177 bash[73044]: ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l 2024-04-25T21:29:17.194 INFO:journalctl@ceph.mds.l.smithi177.stdout:Apr 25 21:29:17 smithi177 podman[73058]: 2024-04-25 21:29:17.086050826 +0000 UTC m=+0.502397490 container remove 29c912c2056b3fee3e8ec09d0971532e745474f293372bc871f897e33c429a4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-1788dafa-0346-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True) 2024-04-25T21:29:17.211 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1788dafa-0346-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T21:29:17.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:29:17.242 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T21:29:17.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:29:17.441 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:19.981 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:29:20.171 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:22.561 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:29:22.748 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:25.103 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:29:25.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:29:25.162 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:29:25.187 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T21:29:25.189 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi037/crash 2024-04-25T21:29:25.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:29:25.232 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:29:25.232 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:29:25.234 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi098/crash 2024-04-25T21:29:25.235 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:29:25.268 INFO:teuthology.orchestra.run.smithi098.stderr:tar: /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:29:25.268 INFO:teuthology.orchestra.run.smithi098.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:29:25.270 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi177/crash 2024-04-25T21:29:25.271 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:29:25.307 INFO:teuthology.orchestra.run.smithi177.stderr:tar: /var/lib/ceph/1788dafa-0346-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:29:25.307 INFO:teuthology.orchestra.run.smithi177.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:29:25.308 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T21:29:25.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T21:29:25.348 INFO:tasks.cephadm:Compressing logs... 2024-04-25T21:29:25.348 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:29:25.391 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:29:25.394 DEBUG:teuthology.orchestra.run.smithi177:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:29:25.417 INFO:teuthology.orchestra.run.smithi098.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:: No such file or directory 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:gzip/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T21:29:25.418 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T21:29:25.422 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.y.log: 90.5% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:29:25.422 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T21:29:25.422 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:29:25.428 INFO:teuthology.orchestra.run.smithi177.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:29:25.428 INFO:teuthology.orchestra.run.smithi177.stderr:: No such file or directory 2024-04-25T21:29:25.428 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T21:29:25.428 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log: -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T21:29:25.428 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.c.log: /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr: 90.5%gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:29:25.429 INFO:teuthology.orchestra.run.smithi177.stderr: -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:29:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:29:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-04-25T21:29:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:29:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T21:29:25.430 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.x.log: /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.a.log: 87.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 90.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T21:29:25.431 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.2% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:29:25.432 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:29:25.432 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.a.log: 94.5% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T21:29:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.h.log: 92.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T21:29:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:29:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T21:29:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:29:25.435 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.i.log: 92.4% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T21:29:25.435 INFO:teuthology.orchestra.run.smithi177.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T21:29:25.435 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T21:29:25.436 INFO:teuthology.orchestra.run.smithi177.stderr: 83.8% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:29:25.436 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:29:25.436 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T21:29:25.441 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:29:25.444 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T21:29:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T21:29:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:29:25.451 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:29:25.452 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T21:29:25.454 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:29:25.455 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T21:29:25.462 INFO:teuthology.orchestra.run.smithi177.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:29:25.464 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T21:29:25.466 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:29:25.467 INFO:teuthology.orchestra.run.smithi177.stderr: 91.0% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:29:25.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:29:25.480 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% 91.0% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:29:25.493 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:29:25.514 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T21:29:25.524 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T21:29:25.534 INFO:teuthology.orchestra.run.smithi098.stderr: 94.7% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T21:29:25.543 INFO:teuthology.orchestra.run.smithi098.stderr: 93.2% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T21:29:25.544 INFO:teuthology.orchestra.run.smithi177.stderr: 93.1% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T21:29:25.548 INFO:teuthology.orchestra.run.smithi177.stderr: 93.1% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T21:29:25.586 INFO:teuthology.orchestra.run.smithi098.stderr: 93.2% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T21:29:25.591 INFO:teuthology.orchestra.run.smithi098.stderr: 94.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T21:29:25.736 INFO:teuthology.orchestra.run.smithi177.stderr: 93.9% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T21:29:25.889 INFO:teuthology.orchestra.run.smithi177.stderr: 91.8% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T21:29:25.918 INFO:teuthology.orchestra.run.smithi037.stderr: 90.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T21:29:26.034 INFO:teuthology.orchestra.run.smithi098.stderr: 91.9% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T21:29:26.459 INFO:teuthology.orchestra.run.smithi037.stderr: 92.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T21:29:26.583 INFO:teuthology.orchestra.run.smithi098.stderr: 91.5% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T21:29:26.839 INFO:teuthology.orchestra.run.smithi098.stderr: 91.8% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T21:29:26.847 INFO:teuthology.orchestra.run.smithi098.stderr: 92.0% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T21:29:26.909 INFO:teuthology.orchestra.run.smithi037.stderr: 90.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T21:29:27.117 INFO:teuthology.orchestra.run.smithi037.stderr: 92.5% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T21:29:27.182 INFO:teuthology.orchestra.run.smithi177.stderr: 93.2% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T21:29:27.296 INFO:teuthology.orchestra.run.smithi037.stderr: 92.6% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T21:29:27.364 INFO:teuthology.orchestra.run.smithi037.stderr: 91.7% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T21:29:27.366 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T21:29:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m1.961s 2024-04-25T21:29:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m8.481s 2024-04-25T21:29:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.278s 2024-04-25T21:29:27.538 INFO:teuthology.orchestra.run.smithi177.stderr: 92.9% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T21:29:27.577 INFO:teuthology.orchestra.run.smithi177.stderr: 92.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T21:29:27.588 INFO:teuthology.orchestra.run.smithi098.stderr: 91.9% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T21:29:27.590 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-04-25T21:29:27.590 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m2.184s 2024-04-25T21:29:27.590 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m6.900s 2024-04-25T21:29:27.590 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.240s 2024-04-25T21:29:27.936 INFO:teuthology.orchestra.run.smithi177.stderr: 92.1% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T21:29:27.979 INFO:teuthology.orchestra.run.smithi177.stderr: 92.3% -- replaced with /var/log/ceph/1788dafa-0346-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T21:29:27.981 INFO:teuthology.orchestra.run.smithi177.stderr: 2024-04-25T21:29:27.981 INFO:teuthology.orchestra.run.smithi177.stderr:real 0m2.574s 2024-04-25T21:29:27.981 INFO:teuthology.orchestra.run.smithi177.stderr:user 0m11.628s 2024-04-25T21:29:27.981 INFO:teuthology.orchestra.run.smithi177.stderr:sys 0m0.407s 2024-04-25T21:29:27.982 INFO:tasks.cephadm:Archiving logs... 2024-04-25T21:29:27.982 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi037/log 2024-04-25T21:29:27.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:29:29.219 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi098/log 2024-04-25T21:29:29.220 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:29:30.310 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi177/log 2024-04-25T21:29:30.311 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:29:32.051 INFO:tasks.cephadm:Removing cluster... 2024-04-25T21:29:32.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force 2024-04-25T21:29:32.252 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:32.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force 2024-04-25T21:29:32.715 INFO:teuthology.orchestra.run.smithi098.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:32.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1788dafa-0346-11ef-bc93-c7b262605968 --force 2024-04-25T21:29:33.178 INFO:teuthology.orchestra.run.smithi177.stdout:Deleting cluster with fsid: 1788dafa-0346-11ef-bc93-c7b262605968 2024-04-25T21:29:33.454 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T21:29:33.455 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:29:33.473 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:29:33.489 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:29:33.504 INFO:tasks.cephadm:Teardown complete 2024-04-25T21:29:33.505 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T21:29:33.516 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T21:29:33.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:29:33.519 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:29:33.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:29:33.636 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T21:29:33.647 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T21:29:33.647 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:29:33.650 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:29:33.653 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:29:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:29:33.665 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:29:33.666 INFO:teuthology.orchestra.run.smithi177.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:29:33.668 INFO:teuthology.orchestra.run.smithi098.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi098.stdout:=============================================================================== 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi098.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 +513us[ +558us] +/- 48ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -1200us[-1155us] +/- 54ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi098.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +615us[ +615us] +/- 62ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi098.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +542us[ +524us] +/- 48ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1172us[-1189us] +/- 54ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +640us[ +622us] +/- 62ms 2024-04-25T21:29:33.669 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +515us[ +515us] +/- 48ms 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1200us[-1200us] +/- 54ms 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +609us[ +609us] +/- 62ms 2024-04-25T21:29:33.670 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:29:33.672 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T21:29:33.682 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T21:29:33.683 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T21:29:33.693 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:29:33.798 DEBUG:teuthology.orchestra.run.smithi098:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:29:33.863 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:29:33.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:29:33.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:33.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:29:33.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:33.985 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:29:34.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:34.012 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T21:29:34.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T21:29:34.034 INFO:teuthology.task.internal:Duration was 2403.896303 seconds 2024-04-25T21:29:34.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T21:29:34.045 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T21:29:34.045 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:29:34.048 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:29:34.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:29:34.084 INFO:teuthology.orchestra.run.smithi098.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:29:34.086 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:29:34.088 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:29:34.456 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T21:29:34.456 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T21:29:34.456 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:29:34.478 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-04-25T21:29:34.478 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:29:34.519 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2024-04-25T21:29:34.520 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:29:34.546 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T21:29:34.546 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:29:34.549 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:29:34.562 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:29:34.611 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T21:29:34.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:29:34.616 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:29:34.634 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:29:35.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T21:29:35.432 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T21:29:35.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:29:35.457 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:29:35.482 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:29:35.509 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T21:29:35.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.526 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.544 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-25T21:29:35.545 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-04-25T21:29:35.574 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2024-04-25T21:29:35.598 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:35.612 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:35.647 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:29:35.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:29:35.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T21:29:35.679 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T21:29:35.679 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi037 2024-04-25T21:29:35.680 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:29:35.734 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi098 2024-04-25T21:29:35.734 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:29:35.769 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673657/remote/smithi177 2024-04-25T21:29:35.769 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:29:35.806 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T21:29:35.806 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:29:35.809 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:29:35.811 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:29:35.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T21:29:35.869 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T21:29:35.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T21:29:35.881 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T21:29:35.881 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:29:35.884 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:29:35.886 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:29:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:29 /home/ubuntu/cephtest 2024-04-25T21:29:35.899 INFO:teuthology.orchestra.run.smithi098.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:29 /home/ubuntu/cephtest 2024-04-25T21:29:35.910 INFO:teuthology.orchestra.run.smithi177.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:29 /home/ubuntu/cephtest 2024-04-25T21:29:35.911 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T21:29:35.922 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T21:29:35.950 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2403.8963027000427 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-25T21:29:35.951 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T21:29:36.040 INFO:teuthology.run:pass